LangGraph(六)——Reflection Agents

Reflection Agents

Reflection是一种提升Agent和类似AI系统质量和成功率的prompt策略，它涉及prompt大型语言模型（LLM）回顾和评判其过去的行为，有时还会结合额外的外部信息，如工具观察结果。

System 2相比与System 1更系统化和并且具有反思性。当正确应用时，Reflection可以帮助大型语言模型（LLM）系统打破System 1的思维模式，更接近表现出System 2类型的行为。

Reflection需要花一些时间，但是使用一些额外的计算资源可以获得更好的输出，但对于知识密集型任务来说，这是值得的，在这些任务中质量比速度更重要。

1. Basic Reflection

  • 组成：由两个LLM调用组成，一个是生成器（generator），尝试直接响应用户请求；另一个是反思器（reflector），扮演教师角色，对初始响应提供建设性评判。
  • 循环：固定次数的循环，最终生成的输出被返回。
Simple Reflection Loop

Generate

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from langchain_fireworks import ChatFireworks
from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are an essay assistant tasked with writing excellent 5-paragraph essays."
            " Generate the best essay possible for the user's request."
            " If the user provides critique, respond with a revised version of your previous attempts.",
        ),
        MessagesPlaceholder(variable_name="messages"),
    ]
)
llm = ChatFireworks(
    model="accounts/fireworks/models/mixtral-8x7b-instruct",
    max_tokens=32768,
)
generate = prompt | llm

Reflect

1
2
3
4
5
6
7
8
9
10
11
reflection_prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a teacher grading an essay submission. Generate critique and recommendations for the user's submission."
            " Provide detailed recommendations, including requests for length, depth, style, etc.",
        ),
        MessagesPlaceholder(variable_name="messages"),
    ]
)
reflect = reflection_prompt | llm

Graph

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
from typing import List, Sequence

from langgraph.graph import END, MessageGraph


async def generation_node(state: Sequence[BaseMessage]):
    return await generate.ainvoke({"messages": state})


async def reflection_node(messages: Sequence[BaseMessage]) -> List[BaseMessage]:
    # Other messages we need to adjust
    cls_map = {"ai": HumanMessage, "human": AIMessage}
    # First message is the original user request. We hold it the same for all nodes
    translated = [messages[0]] + [
        cls_map[msg.type](content=msg.content) for msg in messages[1:]
    ]
    res = await reflect.ainvoke({"messages": translated})
    # We treat the output of this as human feedback for the generator
    return HumanMessage(content=res.content)


builder = MessageGraph()
builder.add_node("generate", generation_node)
builder.add_node("reflect", reflection_node)
builder.set_entry_point("generate")


def should_continue(state: List[BaseMessage]):
    if len(state) > 6:
        # End after 3 iterations
        return END
    return "reflect"


builder.add_conditional_edges("generate", should_continue)
builder.add_edge("reflect", "generate")
graph = builder.compile()

MessageGraph代表一个有状态的图，其中的“状态”仅仅是一系列消息的列表。每次调用Generator或Reflector节点时，它都会在状态的末尾追加一条消息。最终结果由Generator节点返回。

执行

1
2
3
4
5
6
7
8
9
async for event in graph.astream(
    [
        HumanMessage(
            content="Generate an essay on the topicality of The Little Prince and its message in modern life"
        )
    ],
):
    print(event)
    print("---")
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

{'generate': AIMessage(content='Title: The Little Prince: A Timeless Message for Modern Life\n\nIntroduction:\nAntoine de Saint-Exupéry\'s "The Little Prince" is a classic novella that has captured the hearts of millions since its publication in 1943. While it has been over seven decades since its release, the story remains incredibly topical and relevant to modern life. This essay explores the themes of the novel and how they continue to resonate with people in the contemporary world.\n\nBody Paragraph 1: The Relevance of Relationships\nThe Little Prince teaches us about the importance of meaningful relationships in our lives. In the story, the prince leaves his tiny asteroid to explore the universe and meets various inhabitants of other planets, each representing a different aspect of adult existence. These encounters highlight the superficiality of many adult relationships, emphasizing the value of genuine connections. In modern life, where people are often more isolated than ever, despite technological advancements, this message serves as a reminder to prioritize and nurture our real-life relationships.\n\nBody Paragraph 2: The Dangers of Obsession and Materialism\nAnother timely message from The Little Prince is the danger of becoming obsessed with material possessions and success. The novel satirizes the absurdity of the grown-up world, where people are preoccupied with superficial matters, such as the number of baobab trees on a planet. This theme resonates with modern society, where materialism and consumerism have reached unprecedented levels. The Little Prince\'s message encourages us to reassess our priorities and focus on what truly matters in life.\n\nBody Paragraph 3: The Significance of Vulnerability and Emotional Honesty\nThe Little Prince also emphasizes the importance of vulnerability and emotional honesty. Throughout the story, the prince displays a level of emotional intelligence that is both refreshing and thought-provoking. He is not afraid to express his feelings, admit his mistakes, or show his love for others. This message is particularly relevant in today\'s world, where emotional intelligence and mental health are increasingly recognized as essential components of a fulfilling life.\n\nConclusion:\nIn conclusion, The Little Prince remains a topical and powerful work that delivers a timeless message for modern life. Its exploration of relationships, the dangers of obsession and materialism, and the significance of vulnerability and emotional honesty are as relevant today as they were when the book was first published. By embracing the wisdom of The Little Prince, we can better navigate our complex, fast-paced world and lead more meaningful, fulfilling lives.\n\nRevised Essay (if necessary):\n\nTitle: The Little Prince: A Timeless Message for Modern Life\n\nIntroduction:\nAntoine de Saint-Exupéry\'s "The Little Prince" is a timeless novella that has captured the hearts of millions since its publication in 1943. Despite the passage of over seven decades, the story remains incredibly relevant and topical in modern life. This essay explores the novel\'s themes and how they continue to resonate with people in the contemporary world.\n\nBody Paragraph 1: The Importance of Authentic Relationships\nThe Little Prince teaches us about the significance of authentic relationships in our lives. The story highlights the superficiality of many adult relationships, emphasizing the value of genuine connections. This theme is particularly relevant in today\'s world, where people are often more isolated than ever, despite technological advancements. The Little Prince\'s message encourages us to prioritize and nurture our real-life relationships, fostering a sense of community and belonging.\n\nBody Paragraph 2: The Dangers of Materialism and Superficiality\nAnother timely message from The Little Prince is the danger of becoming obsessed with material possessions and superficial success. The novel satirizes the absurdity of the grown-up world, where people are preoccupied with trivial matters. This theme resonates with modern society, where materialism and consumerism have reached unprecedented levels. The Little Prince\'s message encourages us to reassess our priorities and focus on what truly matters in life, such as personal growth, relationships, and self-awareness.\n\nBody Paragraph 3: The Power of Vulnerability and Emotional Intelligence\nThe Little Prince emphasizes the importance of vulnerability and emotional intelligence. Throughout the story, the prince displays a level of emotional intelligence that is both refreshing and thought-provoking. He is not afraid to express his feelings, admit his mistakes, or show his love for others. This message is particularly relevant in today\'s world, where emotional intelligence and mental health are increasingly recognized as essential components of a fulfilling life. By embracing vulnerability, we can build stronger connections and foster a deeper understanding of ourselves and others.\n\nConclusion:\nIn conclusion, The Little Prince remains a topical and powerful work that delivers a timeless message for modern life. Its exploration of authentic relationships, the dangers of materialism and superficiality, and the power of vulnerability and emotional intelligence are as relevant today as they were when the book was first published. By embracing the wisdom of The Little Prince, we can better navigate our complex, fast-paced world and lead more meaningful, fulfilling lives.', response_metadata={'token_usage': {'prompt_tokens': 72, 'total_tokens': 1216, 'completion_tokens': 1144}, 'model_name': 'accounts/fireworks/models/mixtral-8x7b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-8ccbedb3-0c09-4e50-b7aa-fbb233bf4c8b-0')}
---
{'reflect': HumanMessage(content='Your essay on "The Little Prince: A Timeless Message for Modern Life" is well-written, engaging, and thought-provoking. You have done an excellent job of identifying and analyzing the topical themes of the novel and connecting them to contemporary issues. Here are some recommendations to further enhance your essay:\n\n1. Length: While your essay is well-structured and coherent, consider expanding each body paragraph to provide more in-depth analysis. Aim for a minimum of 4-5 developed sentences per paragraph to ensure a thorough exploration of each theme.\n\n2. Depth: To add depth to your essay, consider incorporating specific examples from the novel to support your arguments. For instance, when discussing the importance of authentic relationships, provide a quote or two from the story that illustrates this theme. This will strengthen your argument and demonstrate your understanding of the text.\n\n3. Style: Vary your sentence structure to create a more engaging reading experience. Mix short and long sentences, and use rhetorical questions or transitions to guide the reader through your essay.\n\n4. Recommendation: In your conclusion, consider offering a recommendation or suggestion for applying the lessons from The Little Prince in modern life. This could be a personal anecdote, a current event, or a broader cultural observation that connects the novel\'s themes to contemporary society.\n\n5. Revised Essay: If you decide to revise your essay, consider the following title: "Embracing Vulnerability and Authenticity: The Little Prince\'s Timeless Message for Modern Life." This title better reflects the focus of your essay and highlights the importance of vulnerability and emotional intelligence.\n\nOverall, your essay is engaging and thoughtful. With a few adjustments to length, depth, and style, it can become an even more powerful exploration of the topicality of The Little Prince and its message in modern life.', id='9143f3d4-c27b-44c0-bac0-1e476db7ad4d')}
---
{'generate': AIMessage(content='Title: Embracing Vulnerability and Authenticity: The Little Prince\'s Timeless Message for Modern Life\n\nIntroduction:\nAntoine de Saint-Exupéry\'s "The Little Prince" is a timeless novella that has captured the hearts of millions since its publication in 1943. Despite the passage of over seven decades, the story remains incredibly relevant and topical in modern life. This essay explores the novel\'s themes and how they continue to resonate with people in the contemporary world, focusing on the importance of authentic relationships, the dangers of materialism and superficiality, and the power of vulnerability and emotional intelligence.\n\nBody Paragraph 1: The Importance of Authentic Relationships\nThe Little Prince teaches us about the significance of authentic relationships in our lives. In the story, the prince meets various inhabitants of other planets, each representing a different aspect of adult existence. For instance, the king, the vain man, and the businessman symbolize the superficiality of many adult relationships. However, the fox stands out as a symbol of genuine connections. The fox tells the prince, "One sees clearly only with the heart. Anything essential is invisible to the eyes" (Saint-Exupéry, 1943, p. 65). This quote emphasizes the value of emotional connections and authentic relationships, which are as crucial today as they were in the prince\'s universe.\n\nBody Paragraph 2: The Dangers of Materialism and Superficiality\nAnother timely message from The Little Prince is the danger of becoming obsessed with material possessions and superficial success. The novel satirizes the absurdity of the grown-up world, where people are preoccupied with trivial matters. For example, the businessman believes his purpose is to count stars to "own" them, illustrating the futility of materialism. This theme resonates with modern society, where materialism and consumerism have reached unprecedented levels. The Little Prince\'s message encourages us to reassess our priorities and focus on what truly matters in life, such as personal growth, relationships, and self-awareness.\n\nBody Paragraph 3: The Power of Vulnerability and Emotional Intelligence\nThe Little Prince emphasizes the importance of vulnerability and emotional intelligence. Throughout the story, the prince displays a level of emotional intelligence that is both refreshing and thought-provoking. He is not afraid to express his feelings, admit his mistakes, or show his love for others. This message is particularly relevant in today\'s world, where emotional intelligence and mental health are increasingly recognized as essential components of a fulfilling life. By embracing vulnerability, we can build stronger connections and foster a deeper understanding of ourselves and others.\n\nConclusion:\nIn conclusion, The Little Prince remains a topical and powerful work that delivers a timeless message for modern life. Its exploration of authentic relationships, the dangers of materialism and superficiality, and the power of vulnerability and emotional intelligence are as relevant today as they were when the book was first published. Embracing the wisdom of The Little Prince means acknowledging the importance of genuine connections, reassessing our priorities, and fostering emotional intelligence in our daily lives. By doing so, we can better navigate our complex, fast-paced world and lead more meaningful, fulfilling lives.', response_metadata={'token_usage': {'prompt_tokens': 1627, 'total_tokens': 2351, 'completion_tokens': 724}, 'model_name': 'accounts/fireworks/models/mixtral-8x7b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-753eb78a-8493-42ac-ad50-ba0063cf2686-0')}
---
{'reflect': HumanMessage(content='Your revised essay, "Embracing Vulnerability and Authenticity: The Little Prince\'s Timeless Message for Modern Life," is well-structured and engaging. You have done an excellent job incorporating specific examples from the novel to support your arguments and connecting the themes to contemporary issues. Here are some suggestions for further improvement:\n\n1. Depth: Consider adding more depth to your analysis by discussing the potential consequences of not embracing the lessons from The Little Prince. For instance, what might happen if we continue to prioritize material possessions and superficial success over personal growth and relationships?\n\n2. Style: Experiment with varying your sentence structure and incorporating rhetorical questions to create a more engaging reading experience. For example, in your first body paragraph, you could ask, "What can we learn from the fox\'s wisdom about the significance of authentic relationships?"\n\n3. Recommendation: In your conclusion, consider offering a specific recommendation or suggestion for applying the lessons from The Little Prince in modern life. For instance, you could encourage readers to practice active listening or express gratitude to foster authentic relationships.\n\n4. Citation: Ensure that your citation is formatted correctly according to the required citation style (e.g., MLA, APA, or Chicago). In your current essay, the citation for the quote from the fox is formatted as (Saint-Exupéry, 1943, p. 65), which may not be appropriate for all citation styles.\n\nOverall, your revised essay is engaging, thoughtful, and well-supported. With a few adjustments to depth, style, and recommendations, it can become an even more powerful exploration of the topicality of The Little Prince and its message in modern life.', id='c709ef93-064f-469f-8234-99cdd4649a27')}
---
{'generate': AIMessage(content='Title: Embracing Vulnerability and Authenticity: The Little Prince\'s Timeless Message for Modern Life\n\nIntroduction:\nAntoine de Saint-Exupéry\'s "The Little Prince" is a timeless novella that has captured the hearts of millions since its publication in 1943. Despite the passage of over seven decades, the story remains incredibly relevant and topical in modern life. This essay explores the novel\'s themes and how they continue to resonate with people in the contemporary world, focusing on the importance of authentic relationships, the dangers of materialism and superficiality, and the power of vulnerability and emotional intelligence.\n\nBody Paragraph 1: The Importance of Authentic Relationships\nWhat can we learn from the fox\'s wisdom about the significance of authentic relationships? In the story, the fox tells the prince, "One sees clearly only with the heart. Anything essential is invisible to the eyes" (Saint-Exupéry, 1943, p. 65). Neglecting authentic relationships can lead to feelings of isolation, loneliness, and disconnection in today\'s fast-paced, technology-driven world. By prioritizing genuine connections, we can foster a sense of community, belonging, and emotional well-being.\n\nBody Paragraph 2: The Dangers of Materialism and Superficiality\nAnother timely message from The Little Prince is the danger of becoming obsessed with material possessions and superficial success. The novel satirizes the absurdity of the grown-up world, where people are preoccupied with trivial matters. For example, the businessman believes his purpose is to count stars to "own" them, illustrating the futility of materialism. Prioritizing material possessions and superficial success over personal growth and relationships can lead to feelings of emptiness, dissatisfaction, and a lack of purpose in life.\n\nBody Paragraph 3: The Power of Vulnerability and Emotional Intelligence\nThe Little Prince emphasizes the importance of vulnerability and emotional intelligence. Throughout the story, the prince displays a level of emotional intelligence that is both refreshing and thought-provoking. By embracing vulnerability, we can build stronger connections, communicate more effectively, and foster a deeper understanding of ourselves and others. Neglecting vulnerability and emotional intelligence can lead to misunderstandings, conflict, and damaged relationships.\n\nConclusion:\nIn conclusion, The Little Prince remains a topical and powerful work that delivers a timeless message for modern life. To apply the lessons from The Little Prince in our daily lives, consider practicing active listening, expressing gratitude, and nurturing authentic relationships. By doing so, we can better navigate our complex, fast-paced world and lead more meaningful, fulfilling lives.\n\nReference:\nSaint-Exupéry, A. de. (1943). The Little Prince. Reynal & Hitchcock.', response_metadata={'token_usage': {'prompt_tokens': 2740, 'total_tokens': 3380, 'completion_tokens': 640}, 'model_name': 'accounts/fireworks/models/mixtral-8x7b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-ba992864-5a1b-487d-81a3-a52f74f900ce-0')}
---
{'reflect': HumanMessage(content='Your revised essay, "Embracing Vulnerability and Authenticity: The Little Prince\'s Timeless Message for Modern Life," is well-structured, engaging, and thought-provoking. You have done an excellent job incorporating specific examples from the novel to support your arguments and connecting the themes to contemporary issues. Your essay now includes a clear recommendation for applying the lessons from The Little Prince in modern life.\n\nTo further enhance your essay, consider the following suggestions:\n\n1. Depth: You could add more depth to your analysis by discussing the potential consequences of not embracing the lessons from The Little Prince. For instance, what might happen if we continue to prioritize material possessions and superficial success over personal growth and relationships?\n\n2. Style: Experiment with varying your sentence structure and incorporating rhetorical questions to create a more engaging reading experience. For example, in your first body paragraph, you could ask, "What insights does the fox offer about the significance of authentic relationships?"\n\n3. Quotations: Ensure that your quotations are integrated smoothly into your sentences and are properly formatted according to the required citation style (e.g., MLA, APA, or Chicago).\n\nOverall, your revised essay is engaging, thoughtful, and well-supported. By addressing the importance of authentic relationships, the dangers of materialism and superficiality, and the power of vulnerability and emotional intelligence, you have created a compelling exploration of the topicality of The Little Prince and its message in modern life.', id='deaccdc6-3b38-413e-b996-bb67b90f5e41')}
---
{'generate': AIMessage(content='Title: Embracing Vulnerability and Authenticity: The Little Prince\'s Timeless Message for Modern Life\n\nIntroduction:\nAntoine de Saint-Exupéry\'s "The Little Prince" is a timeless novella that has captured the hearts of millions since its publication in 1943. Despite the passage of over seven decades, the story remains incredibly relevant and topical in modern life. This essay explores the novel\'s themes and how they continue to resonate with people in the contemporary world, focusing on the importance of authentic relationships, the dangers of materialism and superficiality, and the power of vulnerability and emotional intelligence.\n\nBody Paragraph 1: The Importance of Authentic Relationships\nWhat insights does the fox offer about the significance of authentic relationships? In the story, the fox tells the prince, "One sees clearly only with the heart. Anything essential is invisible to the eyes" (Saint-Exupéry, 1943, p. 65). Neglecting authentic relationships can lead to feelings of isolation, loneliness, and disconnection in today\'s fast-paced, technology-driven world. By prioritizing genuine connections, we can foster a sense of community, belonging, and emotional well-being. Ignoring the importance of authentic relationships can result in increased loneliness, mental health issues, and a lack of empathy in society.\n\nBody Paragraph 2: The Dangers of Materialism and Superficiality\nAnother timely message from The Little Prince is the danger of becoming obsessed with material possessions and superficial success. The novel satirizes the absurdity of the grown-up world, where people are preoccupied with trivial matters. For example, the businessman believes his purpose is to count stars to "own" them, illustrating the futility of materialism. Prioritizing material possessions and superficial success over personal growth and relationships can lead to feelings of emptiness, dissatisfaction, and a lack of purpose in life. Continuing to prioritize materialism and superficiality can result in increased inequality, environmental degradation, and a loss of meaning and purpose in our lives.\n\nBody Paragraph 3: The Power of Vulnerability and Emotional Intelligence\nThe Little Prince emphasizes the importance of vulnerability and emotional intelligence. Throughout the story, the prince displays a level of emotional intelligence that is both refreshing and thought-provoking. By embracing vulnerability, we can build stronger connections, communicate more effectively, and foster a deeper understanding of ourselves and others. Neglecting vulnerability and emotional intelligence can lead to misunderstandings, conflict, and damaged relationships. Failing to cultivate vulnerability and emotional intelligence can result in strained relationships, decreased empathy, and an inability to navigate complex emotional situations.\n\nConclusion:\nIn conclusion, The Little Prince remains a topical and powerful work that delivers a timeless message for modern life. To apply the lessons from The Little Prince in our daily lives, consider practicing active listening, expressing gratitude, and nurturing authentic relationships. By doing so, we can better navigate our complex, fast-paced world and lead more meaningful, fulfilling lives.\n\nReference:\nSaint-Exupéry, A. de. (1943). The Little Prince. Reynal & Hitchcock.', response_metadata={'token_usage': {'prompt_tokens': 3718, 'total_tokens': 4448, 'completion_tokens': 730}, 'model_name': 'accounts/fireworks/models/mixtral-8x7b-instruct', 'system_fingerprint': '', 'finish_reason': 'stop', 'logprobs': None}, id='run-6665a113-91c9-4bef-b9b1-3cfc9f33b591-0')}
---
1
print(event['generate'].content)

输出

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Title: Embracing Vulnerability and Authenticity: The Little Prince's Timeless Message for Modern Life

Introduction:
Antoine de Saint-Exupéry's "The Little Prince" is a timeless novella that has captured the hearts of millions since its publication in 1943. Despite the passage of over seven decades, the story remains incredibly relevant and topical in modern life. This essay explores the novel's themes and how they continue to resonate with people in the contemporary world, focusing on the importance of authentic relationships, the dangers of materialism and superficiality, and the power of vulnerability and emotional intelligence.

Body Paragraph 1: The Importance of Authentic Relationships
What insights does the fox offer about the significance of authentic relationships? In the story, the fox tells the prince, "One sees clearly only with the heart. Anything essential is invisible to the eyes" (Saint-Exupéry, 1943, p. 65). Neglecting authentic relationships can lead to feelings of isolation, loneliness, and disconnection in today's fast-paced, technology-driven world. By prioritizing genuine connections, we can foster a sense of community, belonging, and emotional well-being. Ignoring the importance of authentic relationships can result in increased loneliness, mental health issues, and a lack of empathy in society.

Body Paragraph 2: The Dangers of Materialism and Superficiality
Another timely message from The Little Prince is the danger of becoming obsessed with material possessions and superficial success. The novel satirizes the absurdity of the grown-up world, where people are preoccupied with trivial matters. For example, the businessman believes his purpose is to count stars to "own" them, illustrating the futility of materialism. Prioritizing material possessions and superficial success over personal growth and relationships can lead to feelings of emptiness, dissatisfaction, and a lack of purpose in life. Continuing to prioritize materialism and superficiality can result in increased inequality, environmental degradation, and a loss of meaning and purpose in our lives.

Body Paragraph 3: The Power of Vulnerability and Emotional Intelligence
The Little Prince emphasizes the importance of vulnerability and emotional intelligence. Throughout the story, the prince displays a level of emotional intelligence that is both refreshing and thought-provoking. By embracing vulnerability, we can build stronger connections, communicate more effectively, and foster a deeper understanding of ourselves and others. Neglecting vulnerability and emotional intelligence can lead to misunderstandings, conflict, and damaged relationships. Failing to cultivate vulnerability and emotional intelligence can result in strained relationships, decreased empathy, and an inability to navigate complex emotional situations.

Conclusion:
In conclusion, The Little Prince remains a topical and powerful work that delivers a timeless message for modern life. To apply the lessons from The Little Prince in our daily lives, consider practicing active listening, expressing gratitude, and nurturing authentic relationships. By doing so, we can better navigate our complex, fast-paced world and lead more meaningful, fulfilling lives.

Reference:
Saint-Exupéry, A. de. (1943). The Little Prince. Reynal & Hitchcock.

LangSmith流程展示：

这种简单类型的Reflect有时可以通过让大型语言模型（LLM）多次尝试优化其输出，并通过让Reflector节点在评判输出时采用不同的角色来提高性能。然而，由于Reflect步骤没有基于任何外部过程，最终结果可能并不比原始结果有显著的改进。

2. Reflexion

  • Reflexion：一种通过口头反馈和自我反思学习的结构。
  • 特点：代理明确评判每个响应，并将评判基于外部数据，生成引用，明确指出生成响应中多余和缺失的部分。这使得反思的内容更具建设性，并更好地引导生成器响应反馈。
  • 循环逻辑：可以在固定步骤后停止，也可将循环决策交给Reflect LLM。

Reflexion Agent 流程概览：

Reflexion Actor Overview

对于每一步，Responder的任务是生成一个响应，同时以搜索查询的形式进行额外的操作。然后，Revisor对当前状态进行反思。

Tool

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
from langchain_community.tools.tavily_search import TavilySearchResults
from langchain_community.utilities.tavily_search import TavilySearchAPIWrapper
from collections import defaultdict
from typing import List
import json


from langchain.output_parsers.openai_tools import (
    JsonOutputToolsParser,
    PydanticToolsParser,
)
from langchain_core.messages import AIMessage, BaseMessage, HumanMessage, ToolMessage
from langgraph.prebuilt.tool_executor import ToolExecutor, ToolInvocation


search = TavilySearchAPIWrapper()
tavily_tool = TavilySearchResults(api_wrapper=search, max_results=5)


tool_executor = ToolExecutor([tavily_tool])
parser = JsonOutputToolsParser(return_id=True)


def execute_tools(state: List[BaseMessage]) -> List[BaseMessage]:
    tool_invocation: AIMessage = state[-1]
    parsed_tool_calls = parser.invoke(tool_invocation)
    ids = []
    tool_invocations = []
    for parsed_call in parsed_tool_calls:
        for query in parsed_call["args"]["search_queries"]:
            tool_invocations.append(
                ToolInvocation(
                    # We only have this one for now. Would want to map it
                    # if we change
                    tool="tavily_search_results_json",
                    tool_input=query,
                )
            )
            ids.append(parsed_call["id"])

    outputs = tool_executor.batch(tool_invocations)
    outputs_map = defaultdict(dict)
    for id_, output, invocation in zip(ids, outputs, tool_invocations):
        outputs_map[id_][invocation.tool_input] = output

    return [
        ToolMessage(content=json.dumps(query_outputs), tool_call_id=id_)
        for id_, query_outputs in outputs_map.items()
    ]

Responder

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
import datetime

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.pydantic_v1 import BaseModel, Field, ValidationError
from langchain_openai import ChatOpenAI
from langsmith import traceable

actor_prompt_template = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            """You are expert researcher.
Current time: {time}

1. {first_instruction}
2. Reflect and critique your answer. Be severe to maximize improvement.
3. Recommend search queries to research information and improve your answer.""",
        ),
        MessagesPlaceholder(variable_name="messages"),
        ("system", "Answer the user's question above using the required format."),
    ]
).partial(
    time=lambda: datetime.datetime.now().isoformat(),
)


class Reflection(BaseModel):
    missing: str = Field(description="Critique of what is missing.")
    superfluous: str = Field(description="Critique of what is superfluous")


class AnswerQuestion(BaseModel):
    """Answer the question."""

    answer: str = Field(description="~250 word detailed answer to the question.")
    reflection: Reflection = Field(description="Your reflection on the initial answer.")
    search_queries: List[str] = Field(
        description="1-3 search queries for researching improvements to address the critique of your current answer."
    )


llm = ChatOpenAI(model="gpt-4-turbo")
initial_answer_chain = actor_prompt_template.partial(
    first_instruction="Provide a detailed ~250 word answer."
) | llm.bind_tools(tools=[AnswerQuestion], tool_choice="AnswerQuestion")
validator = PydanticToolsParser(tools=[AnswerQuestion])


class ResponderWithRetries:
    def __init__(self, runnable, validator):
        self.runnable = runnable
        self.validator = validator

    @traceable
    def respond(self, state: List[BaseMessage]):
        response = []
        for attempt in range(3):
            try:
                response = self.runnable.invoke({"messages": state})
                self.validator.invoke(response)
                return response
            except ValidationError as e:
                state = state + [HumanMessage(content=repr(e))]
        return response
    
first_responder = ResponderWithRetries(
    runnable=initial_answer_chain, validator=validator
)

Revision

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
revise_instructions = """Revise your previous answer using the new information.
    - You should use the previous critique to add important information to your answer.
        - You MUST include numerical citations in your revised answer to ensure it can be verified.
        - Add a "References" section to the bottom of your answer (which does not count towards the word limit). In form of:
            - [1] https://example.com
            - [2] https://example.com
    - You should use the previous critique to remove superfluous information from your answer and make SURE it is not more than 250 words.
"""


# Extend the initial answer schema to include references.
# Forcing citation in the model encourages grounded responses
class ReviseAnswer(AnswerQuestion):
    """Revise your original answer to your question."""

    references: List[str] = Field(
        description="Citations motivating your updated answer."
    )


revision_chain = actor_prompt_template.partial(
    first_instruction=revise_instructions
) | llm.bind_tools(tools=[ReviseAnswer], tool_choice="ReviseAnswer")
revision_validator = PydanticToolsParser(tools=[ReviseAnswer])

revisor = ResponderWithRetries(runnable=revision_chain, validator=revision_validator)

构造Graph

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
from langgraph.graph import END, MessageGraph

MAX_ITERATIONS = 5
builder = MessageGraph()
builder.add_node("draft", first_responder.respond)
builder.add_node("execute_tools", execute_tools)
builder.add_node("revise", revisor.respond)
# draft -> execute_tools
builder.add_edge("draft", "execute_tools")
# execute_tools -> revise
builder.add_edge("execute_tools", "revise")

# Define looping logic:


def _get_num_iterations(state: List[BaseMessage]):
    i = 0
    for m in state[::-1]:
        if not isinstance(m, (ToolMessage, AIMessage)):
            break
        i += 1
    return i


def event_loop(state: List[BaseMessage]) -> str:
    # in our case, we'll just stop after N plans
    num_iterations = _get_num_iterations(state)
    if num_iterations > MAX_ITERATIONS:
        return END
    return "execute_tools"


# revise -> execute_tools OR end
builder.add_conditional_edges("revise", event_loop)
builder.set_entry_point("draft")
graph = builder.compile()

执行

1
2
3
4
5
6
7
8
events = graph.stream(
    [HumanMessage(content="How should we handle the climate crisis?")]
)
for i, step in enumerate(events):
    node, output = next(iter(step.items()))
    print(f"## {i+1}. {node}")
    print(str(output)[:100] + " ...")
    print("---")

输出

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
## 1. draft
content='' additional_kwargs={'tool_calls': [{'id': 'call_zsSoRPEYQbdMCfTFE6Zt3yWZ', 'function': {'a ...
---
## 2. execute_tools
[ToolMessage(content='{"role of international cooperation in climate crisis": [{"url": "https://www. ...
---
## 3. revise
content='' additional_kwargs={'tool_calls': [{'id': 'call_haETlmZsvMvTDgvVnoX7PF8B', 'function': {'a ...
---
## 4. execute_tools
[ToolMessage(content='{"effective climate change policies 2024": [{"url": "https://www.nature.com/ar ...
---
## 5. revise
content='' additional_kwargs={'tool_calls': [{'id': 'call_ftYfUYHV6J6PQP56lTvAgQAa', 'function': {'a ...
---
## 6. execute_tools
[ToolMessage(content='{"global climate agreements effectiveness": [{"url": "https://www.climatereali ...
---
## 7. revise
content='' additional_kwargs={'tool_calls': [{'id': 'call_PgTBJqyP5dbBEANOpRIcLaPg', 'function': {'a ...
---

结果

1
print(parser.invoke(step['revise'])[-1]["args"]["answer"])

输出

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Addressing the climate crisis requires a comprehensive and multifaceted approach. Here are several key strategies:

1. **Transition to Renewable Energy:** Shifting from fossil fuels to renewable energy sources like solar, wind, and hydroelectric is crucial. This reduces greenhouse gas emissions, which are primary drivers of climate change [1].

2. **Enhance Energy Efficiency:** Improving energy efficiency in buildings, vehicles, and industries can significantly reduce energy consumption and emissions [1].

3. **Carbon Pricing:** Implementing carbon pricing mechanisms, such as carbon taxes or cap-and-trade systems, incentivizes companies to reduce their carbon footprint [1].

4. **Conservation and Reforestation:** Protecting forests and other ecosystems can absorb CO2 from the atmosphere. Reforestation and afforestation efforts are also vital, as they sequester atmospheric carbon and enhance biodiversity [1].

5. **Climate Adaptation Measures:** Developing infrastructure and policies to adapt to climate impacts, like rising sea levels and increased weather variability, is essential. This includes building flood defenses, creating drought-resistant crops, and improving water management systems [1].

6. **International Cooperation:** Effective management of the climate crisis requires global cooperation. Agreements like the Paris Agreement are essential as they engage countries in a united effort to curb emissions and adapt to climate changes [2].

7. **Education and Public Awareness:** Raising awareness and educating the public about the impacts of climate change and the importance of mitigation efforts is crucial. This can drive behavioral changes and support for policy measures aimed at combating climate change [3].

LangSmith流程展示：

这种Agent能够有效地使用明确的反思和基于网络搜索的内容来提高最终响应的质量。然而，它过程比较固定，如果它犯了一个错误，那个错误可能会影响后续的决策。

3. Language Agent Tree Search(LATS)

LATS：一种通用的LLM Agent搜索算法，它结合了反思/评估和搜索（特别是蒙特卡洛树搜索），以实现比类似技术如ReACT、Reflexion甚至思维树更好的整体任务性能。它采用标准的强化学习（RL）任务框架，将RL Agents、价值函数和优化器全部替换为对LLM的调用。这可以帮助Agent适应和解决复杂任务，避免陷入重复循环。

LATS 流程概览：

搜索主要步骤：
1. Select(选择)：依据第二步累计奖励来挑选挑选最佳的下一步 action，如果找到了解决方案或达到了最大搜索深度则返回响应，否则继续搜索
2. Expand and simulate(扩展和模拟): 生成N个可能的 action，并行执行它们。
3. Reflect + evaluate(反思+评估)：观察这些action的结果，并根据反思（以及可能的外部反馈）给决策打分。
4. Backpropagate(反向传播)：根据结果更新根轨迹的得分。

如果 Agent 通过高质量的环境奖励或可靠的反思得分反馈循环，搜索能够准确地区分不同的行动轨迹并选择最佳路径。然后，最终的轨迹可以保存到外部存储器中（或用于模型微调），以在未来改进模型。

Select步骤挑选最高上置信界限（UCT）的节点，这样可以在预期奖励（第一项）与探索新路径的激励（第二项）之间取得平衡。
​

Graph State

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
from __future__ import annotations
from typing_extensions import TypedDict
import math
from typing import List, Optional
from langchain_core.messages import AIMessage, BaseMessage, HumanMessage, ToolMessage


class Node:
    def __init__(
        self,
        messages: List[BaseMessage],
        reflection: Reflection,
        parent: Optional[Node] = None,
    ):
        self.messages = messages
        self.parent = parent
        self.children = []
        self.value = 0
        self.visits = 0
        self.reflection = reflection
        self.depth = parent.depth + 1 if parent is not None else 1
        self._is_solved = reflection.found_solution if reflection else False
        if self._is_solved:
            self._mark_tree_as_solved()
        self.backpropagate(reflection.normalized_score)

    def __repr__(self) -> str:
        return (
            f"<Node value={self.value}, visits={self.visits},"
            f" solution={self.messages} reflection={self.reflection}/>"
        )

    @property
    def is_solved(self):
        """If any solutions exist, we can end the search."""
        return self._is_solved

    @property
    def is_terminal(self):
        return not self.children

    @property
    def best_child(self):
        """Select the child with the highest UCT to search next."""
        if not self.children:
            return None
        all_nodes = self._get_all_children()
        return max(all_nodes, key=lambda child: child.upper_confidence_bound())

    @property
    def best_child_score(self):
        """Return the child with the highest value."""
        if not self.children:
            return None
        return max(self.children, key=lambda child: int(child.is_solved) * child.value)

    @property
    def height(self) -> int:
        """Check for how far we've rolled out the tree."""
        if self.children:
            return 1 + max([child.height for child in self.children])
        return 1

    def upper_confidence_bound(self, exploration_weight=1.0):
        """Return the UCT score. This helps balance exploration vs. exploitation of a branch."""
        if self.parent is None:
            raise ValueError("Cannot obtain UCT from root node")
        if self.visits == 0:
            return self.value
        # Encourages exploitation of high-value trajectories
        average_reward = self.value / self.visits
        # Encourages exploration of less-visited trajectories
        exploration_term = math.sqrt(math.log(self.parent.visits) / self.visits)
        return average_reward + exploration_weight * exploration_term

    def backpropagate(self, reward: float):
        """Update the score of this node and its parents."""
        node = self
        while node:
            node.visits += 1
            node.value = (node.value * (node.visits - 1) + reward) / node.visits
            node = node.parent

    def get_messages(self, include_reflections: bool = True):
        if include_reflections:
            return self.messages + [self.reflection.as_message()]
        return self.messages

    def get_trajectory(self, include_reflections: bool = True) -> List[BaseMessage]:
        """Get messages representing this search branch."""
        messages = []
        node = self
        while node:
            messages.extend(
                node.get_messages(include_reflections=include_reflections)[::-1]
            )
            node = node.parent
        # Reverse the final back-tracked trajectory to return in the correct order
        return messages[::-1]  # root solution, reflection, child 1, ...

    def _get_all_children(self):
        all_nodes = []
        nodes = deque()
        nodes.append(self)
        while nodes:
            node = nodes.popleft()
            all_nodes.extend(node.children)
            for n in node.children:
                nodes.append(n)
        return all_nodes

    def get_best_solution(self):
        """Return the best solution from within the current sub-tree."""
        all_nodes = [self] + self._get_all_children()
        best_node = max(
            all_nodes,
            # We filter out all non-terminal, non-solution trajectories
            key=lambda node: int(node.is_terminal and node.is_solved) * node.value,
        )
        return best_node

    def _mark_tree_as_solved(self):
        parent = self.parent
        while parent:
            parent._is_solved = True
            parent = parent.parent
            

class TreeState(TypedDict):
    # The full tree
    root: Node
    # The original input
    input: str

Tools And Model

1
2
3
4
5
6
7
8
9
10
11
12
13
from langchain_openai import ChatOpenAI

from langchain_community.tools.tavily_search import TavilySearchResults
from langchain_community.utilities.tavily_search import TavilySearchAPIWrapper

from langgraph.prebuilt.tool_executor import ToolExecutor, ToolInvocation

llm = ChatOpenAI(model="gpt-4-turbo")

search = TavilySearchAPIWrapper()
tavily_tool = TavilySearchResults(api_wrapper=search, max_results=5)
tools = [tavily_tool]
tool_executor = ToolExecutor(tools=tools)

Reflection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
from langchain.chains import create_structured_output_runnable
from langchain.output_parsers.openai_tools import (
    JsonOutputToolsParser,
    PydanticToolsParser,
)
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.runnables import chain as as_runnable


class Reflection(BaseModel):
    reflections: str = Field(
        description="The critique and reflections on the sufficiency, superfluency,"
        " and general quality of the response"
    )
    score: int = Field(
        description="Score from 0-10 on the quality of the candidate response.",
        gte=0,
        lte=10,
    )
    found_solution: bool = Field(
        description="Whether the response has fully solved the question or task."
    )

    def as_message(self):
        return HumanMessage(
            content=f"Reasoning: {self.reflections}\nScore: {self.score}"
        )

    @property
    def normalized_score(self) -> float:
        return self.score / 10.0


prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "Reflect and grade the assistant response to the user question below.",
        ),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="candidate"),
    ]
)

reflection_llm_chain = (
    prompt
    | llm.bind_tools(tools=[Reflection], tool_choice="Reflection").with_config(
        run_name="Reflection"
    )
    | PydanticToolsParser(tools=[Reflection])
)


@as_runnable
def reflection_chain(inputs) -> Reflection:
    tool_choices = reflection_llm_chain.invoke(inputs)
    reflection = tool_choices[0]
    if not isinstance(inputs["candidate"][-1], AIMessage):
        reflection.found_solution = False
    return reflection

Response

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
from typing import List

from langchain_core.prompt_values import ChatPromptValue
from langchain_core.pydantic_v1 import BaseModel, Field, ValidationError
from langchain_core.runnables import RunnableConfig

prompt_template = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are an AI assistant.",
        ),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="messages", optional=True),
    ]
)


initial_answer_chain = prompt_template | llm.bind_tools(tools=tools).with_config(
    run_name="GenerateInitialCandidate"
)


parser = JsonOutputToolsParser(return_id=True)

Graph Nodes

Starting Node

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import json


# Define the node we will add to the graph
def generate_initial_response(state: TreeState) -> dict:
    """Generate the initial candidate response."""
    res = initial_answer_chain.invoke({"input": state["input"]})
    parsed = parser.invoke(res)
    tool_responses = tool_executor.batch(
        [ToolInvocation(tool=r["type"], tool_input=r["args"]) for r in parsed]
    )
    output_messages = [res] + [
        ToolMessage(content=json.dumps(resp), tool_call_id=tool_call["id"])
        for resp, tool_call in zip(tool_responses, parsed)
    ]
    reflection = reflection_chain.invoke(
        {"input": state["input"], "candidate": output_messages}
    )
    root = Node(output_messages, reflection=reflection)
    return {
        **state,
        "root": root,
    }

Generation Node

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

from collections import defaultdict, deque
# This generates N candidate values
# for a single input to sample actions from the environment


def generate_candidates(messages: ChatPromptValue, config: RunnableConfig):
    n = config["configurable"].get("N", 5)
    bound_kwargs = llm.bind_tools(tools=tools).kwargs
    chat_result = llm.generate(
        [messages.to_messages()],
        n=n,
        callbacks=config["callbacks"],
        run_name="GenerateCandidates",
        **bound_kwargs
    )
    return [gen.message for gen in chat_result.generations[0]]


expansion_chain = prompt_template | generate_candidates



def expand(state: TreeState, config: RunnableConfig) -> dict:
    """Starting from the "best" node in the tree, generate N candidates for the next step."""
    root = state["root"]
    best_candidate: Node = root.best_child if root.children else root
    messages = best_candidate.get_trajectory()
    # Generate N candidates from the single child candidate
    new_candidates = expansion_chain.invoke(
        {"input": state["input"], "messages": messages}, config
    )
    parsed = parser.batch(new_candidates)
    flattened = [
        (i, tool_call)
        for i, tool_calls in enumerate(parsed)
        for tool_call in tool_calls
    ]
    tool_responses = tool_executor.batch(
        [
            ToolInvocation(tool=tool_call["type"], tool_input=tool_call["args"])
            for _, tool_call in flattened
        ]
    )
    collected_responses = defaultdict(list)
    for (i, tool_call), resp in zip(flattened, tool_responses):
        collected_responses[i].append(
            ToolMessage(content=json.dumps(resp), tool_call_id=tool_call["id"])
        )
    output_messages = []
    for i, candidate in enumerate(new_candidates):
        output_messages.append([candidate] + collected_responses[i])

    # Reflect on each candidate
    # For tasks with external validation, you'd add that here.
    reflections = reflection_chain.batch(
        [{"input": state["input"], "candidate": msges} for msges in output_messages],
        config,
    )
    # Grow tree
    child_nodes = [
        Node(cand, parent=best_candidate, reflection=reflection)
        for cand, reflection in zip(output_messages, reflections)
    ]
    best_candidate.children.extend(child_nodes)
    # We have already extended the tree directly, so we just return the state
    return state

创建Graph

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
from langgraph.graph import END, StateGraph


def should_loop(state: TreeState):
    """Determine whether to continue the tree search."""
    root = state["root"]
    if root.is_solved:
        return END
    if root.height > 5:
        return END
    return "expand"


builder = StateGraph(TreeState)
builder.add_node("start", generate_initial_response)
builder.add_node("expand", expand)
builder.set_entry_point("start")


builder.add_conditional_edges(
    "start",
    # Either expand/rollout or finish
    should_loop,
)
builder.add_conditional_edges(
    "expand",
    # Either continue to rollout or finish
    should_loop,
)

graph = builder.compile()

执行

1
2
3
4
5
6
question = "Generate a table with the average size and weight, as well as the oldest recorded instance for each of the top 5 most common birds."
for step in graph.stream({"input": question}):
    step_name, step_state = next(iter(step.items()))
    print(step_name)
    print("rolled out: ", step_state["root"].height)
    print("---")

输出

1
2
3
4
5
6
start
rolled out:  1
---
expand
rolled out:  2
---

结果

1
2
3
solution_node=step['expand']["root"].get_best_solution()
best_trajectory = solution_node.get_trajectory(include_reflections=False)
print(best_trajectory[-1].content)

输出

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Here's a summary table including the average size and weight, as well as the oldest recorded age for each of the top 5 most common birds based on the gathered data. Note that exact average sizes and weights for some birds were not directly found, so typical ranges are provided based on available information.

| Bird Species         | Average Size (Length) | Average Weight  | Oldest Recorded Age |
|----------------------|-----------------------|-----------------|---------------------|
| Northern Cardinal    | 8.3 - 9.1 inches      | 42 - 48 grams   | 15 years (approx.)  |
| American Robin       | 9 - 11 inches         | 77 - 85 grams   | 13 years (approx.)  |
| Mourning Dove        | 9 - 13 inches         | 96 - 170 grams  | 31 years            |
| American Crow        | 16 - 21 inches        | 316 - 620 grams | 29 years (approx.)  |
| European Starling    | 7.5 - 9 inches        | 60 - 96 grams   | 23 years (approx.)  |

### Notes:
1. **Northern Cardinal**: Known for its bright red coloring in males, this bird is common across the eastern and central United States.
2. **American Robin**: Recognizable by its red chest, this bird is widespread across North America.
3. **Mourning Dove**: Known for its mournful cooing, these birds are found throughout North America.
4. **American Crow**: A large, intelligent bird found across most of the United States.
5. **European Starling**: Introduced in the U.S. in the 19th century, it has now spread widely.

The oldest recorded ages provided above are based on available data, but actual longevity may vary widely in wild populations.

LangSmith流程展示：

LATS统一了其他Agent架构的推理、规划和反思组件，如Reflexion、Tree of Thoughts和plan-and-execute Agent。LATS还利用反思和基于环境的反馈的反向传播来改进搜索过程。尽管它可能对奖励分数敏感，这个通用算法可以灵活地应用于各种任务。

LATS与其他架构对比

4. 结论

上述所有技术都利用了额外的LLM推理，以增加生成更高质量输出的可能性，或正确响应更复杂的推理任务。虽然这需要额外的时间，但当输出质量比响应时间更重要时，这样做是可行的。如果将轨迹保存起来（或作为 fine-tuning data），就可以更新模型以避免重复的错误。

官方资源

代码示例更详细说明：

LangGraph
#LLM #LLM学习笔记 #Agent #LangGraph
LangGraph(六)——Reflection Agents
https://mztchaoqun.com.cn/posts/D37_Refection_Agents/
作者
mztchaoqun
发布于
2024年9月13日
许可协议