By

欢迎转载，请支持原创，保留原文链接:blog.ilibrary.me

问题和解决方法

在提取内容并且要求openai输出为json格式的时候，遇到openai不停重复空记录的问题, 往json数组里面添加重复的没有意义的记录, 最终导致输出的token太多，超出了max tokens设置，json字符串被截断，客户端没法正常解析json.

不断重复没有意义的记录：

[{ "program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},

像是prompt触发了Transfer decoder的某个死循环. GPT是一个一个字符生成的，每个时刻生成的内容会被填进GPU重新推理下一个字符，输入重复的字串，输出重复的内容，听起来很符合计算机的逻辑，进入死循环。像是计算机的思维钻入了牛角尖。

具体现象可以看下文章尾部的截图和日志(截图和日志分属两个不同的请求，问题是一样的):

出问题的prompt:

{'content': '------\n\n- **Application Deadlines**\nRepresents the application deadlines for an undergraduate program, covering different types of deadlines such as early action, regular decision, and rolling admissions.\n  - **First-time Student Recommended FAFSA Filing Deadline**: For the 2025-26 academic year, it is recommended to file as soon as possible after the full application rollout.\n  - **Current Students Recommended FAFSA Filing Deadline**: April 15 for the 2025-26 academic year.\n  - **PA State Grant FAFSA Filing Deadline**: May 1 for the 2025-26 academic year.', 
'instruction': ('- Clearly specify all relevant deadlines, including early decision, regular decision, and rolling admission dates.\n- Ensure only deadlines for the specified program are included, excluding those for other programs.\n- Distinguish between concurrent/dual degrees and single degrees, ensuring the two are not confused',), 
'category': 'undergraduate program', 'category_tag': 'UNDERGRADUATE_PROGRAM', 'university': 'Pennsylvania State University', 'degree': 'COMMUNICATION ARTS AND SCIENCES B.A.'}

解决办法:

指定要求openai不要输出空的内容。 注意最后添加的一条指令，这条指令能引导OpenAI钻出牛角尖。

application_deadlines_extract_instruction = (
    "- Clearly specify all relevant deadlines, including early decision, regular decision, and rolling admission dates.\n"
    "- Ensure only deadlines for the specified program are included, excluding those for other programs.\n"
    "- Distinguish between concurrent/dual degrees and single degrees, ensuring the two are not confused"
    "- If the extracted date is empty, don't include it in the json array." # Add this line, to resolve openai infinite loop error, 
    ),

延生

关于数据重复，还存在一个训练数据leak的有趣现象, ChatGPT狂吐训练数据，还带个人信息：DeepMind发现大bug引争议, 算是一种GPT攻击方式。简单的理解就是GPT不停地重复某些数据的时候会匹配到训练数据中的一段文本，然后这段文本及后面的训练数据就会被匹配上，然后原封不动的吐出来。

问题截图

问题截图如下:

日志：

 File "/opt/anaconda3/envs/applyai-coredata/lib/python3.11/site-packages/llama_index/program/openai/base.py", line 67, in _parse_tool_calls
    output = output_cls.parse_raw(function_call.arguments)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pydantic/main.py", line 548, in pydantic.main.BaseModel.parse_raw
pydantic.error_wrappers.ValidationError: 1 validation error for ApplicationDeadlines
__root__
  Unterminated string starting at: line 1 column 12402 (char 12401) (type=value_error.jsondecode; msg=Unterminated string starting at; doc={"deadlines":[{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program":"COMMUNICATION ARTS AND SCIENCES B.A.","category":"","admit_term":""},{"date":"","program; pos=12401; lineno=1; colno=12402)
2024-12-04 11:08:43,052 INFO - 0000..0000 openai_pydantic_program_proxy.acall:91: Error in acall with args: ()
kwargs:{'content': '------\n\n- **Application Deadlines**\nRepresents the application deadlines for an undergraduate program, covering different types of deadlines such as early action, regular decision, and rolling admissions.\n  - **First-time Student Recommended FAFSA Filing Deadline**: For the 2025-26 academic year, it is recommended to file as soon as possible after the full application rollout.\n  - **Current Students Recommended FAFSA Filing Deadline**: April 15 for the 2025-26 academic year.\n  - **PA State Grant FAFSA Filing Deadline**: May 1 for the 2025-26 academic year.', 'instruction': ('- Clearly specify all relevant deadlines, including early decision, regular decision, and rolling admission dates.\n- Ensure only deadlines for the specified program are included, excluding those for other programs.\n- Distinguish between concurrent/dual degrees and single degrees, ensuring the two are not confused',), 'category': 'undergraduate program', 'category_tag': 'UNDERGRADUATE_PROGRAM', 'university': 'Pennsylvania State University', 'degree': 'COMMUNICATION ARTS AND SCIENCES B.A.'}