炒冷饭系列【一】斯坦福AI小镇
最近突发奇想把玩了一下斯坦福大学开源的generative_agents项目:https://github.com/joonspk-research/generative_agents
没什么其他原因,就是觉得这种llm的应用方式很有趣味。代码读了一遍之后发现bug甚多,还有一些像是尚未完工的临时代码,看的十分痛苦,所以干脆重构了一版,使用白嫖的千帆平台Yi-34B-Chat进行测试,效果基本和原始项目相同(准确地讲效果上缺乏评估体系,基本就是肉眼评估,即:每个agent可以思考、行动、交流,并且看起来没那么智障)。简单记录一下重构过程中的所思所想,想看一下效果的朋友可以跳到5.Quick Start一节。
项目放在:https://github.com/Archermmt/wounderland
注意: 因为使用了llamaindex,没有翻墙的情况下启动时间极长,建议先使用科学上网再运行wounderland,agent的长期记忆这部分正在考虑重构,llamaindex的设计真的是一言难尽,后文会有专门吐槽。
项目采用前后端分离方式,前端使用原作中相同的phaser作为游戏engine,同时用了vue来做信息传递。后端使用python搭建,调用llm服务作为agent的核心部分,并使用llamaindex作为长期记忆存储,基本拼凑出了agent的相关模块,把玩之后乍一看挺有意思,但仔细思考发现存在一些问题,一并记录在此。
1. 整体架构
repo分为两部分,playground为基于django搭建的前端框架,主要用于呈现效果;wounderland则是围绕llm构建的agent框架,用于实现memory、percept、planning、reflect等相关模块,后续会尝试大模型本地部署来解决实时反馈的问题。
作为agent的开山之作,原始项目基本确定了agent的整体框架:
因为小镇中agent只做互相交流和行程规划,所以各种tools基本没有用到,唯一需要的tool就是path_finder用于寻路。框架的其余部分基本都用到了,可以说麻雀虽小五脏俱全,用来理解agent框架再好不过,以下按照各个部分进行分析。
2.前端引擎
基本照抄原作中的phaser使用方式,assets图片和tilemap部分也是直接拿来主义,因为自己实在不擅长做美工TOT。。。。不过这个游戏资产部分也可以使用AIGC的方式进行生成,例如generative_agents的升级版本https://github.com/a16z-infra/ai-town 就已经可以自由生成游戏素材了,后面有时间了会研究一下。
前端代码都在playground目录下,大致划分为三个部分:
2.1 server
由django搭建,由于目前还没有引入太多交互和数据,所以就简单用django创建了几个models,分别用于记录用户信息(LLM的keys)和Agent状态信息(用于从某一个时间点加载agent状态,类似游戏save/load功能),开放了几个url分别用于和游戏引擎交互:
urlpatterns = [
url("village", game_views.village), //主界面,没有绑定view,不要访问
url("start_game", game_views.start_game, name="start_game"), //初始化游戏
url("agent_think", game_views.agent_think, name="agent_think"), //agent开始思考,按照配置中的思考时间间隔自动触发
url("agent_save", game_views.agent_save, name="agent_save"), //保存agent状态,按照配置的时间间隔记录
url("get_time", game_views.get_time, name="get_time"), //更新游戏时间
url("user_login", game_views.user_login, name="user_login"), //用户登录
url("user_logout", game_views.user_logout, name="user_logout"), //用户注销
url("user_add_key", game_views.user_add_key, name="user_add_key"), //添加llm keys(需要用户已经登陆)
]
2.2 页面
用了phaser+vue的结构搭建,phaser的实现和游戏素材基本和原作相同,做了一点小修改。
1.可以用鼠标进行放缩(滚轮)和拖拽(点击左键拖拽)
2.增加了一个看板系统(board),可以显示当前agent和user的信息,用来配置llm keys以及监控agent状态,找到了一个红白机风格的css项目:https://github.com/nostalgic-css/NES.css,和界面风格还是很搭的。
具体效果如下:
2.3 前端实现
使用javascript开发,鉴于原作把所有js代码直接嵌入到了html中,不利于维护,wounderland对此部分架构进行重构,分成几部分:
1.agent类:继承Phaser.GameObjects.Sprite,用于呈现agent并与后端进行交互,主要交互就是调用agent_think让agent进行感知和规划
2.land类:继承Phaser.Scene,用于呈现小镇地图,并和board进行信息交互,添加了一个可缩放、拖拽和跟踪agent的相机(点击agent看板的follow即可跟踪)
3.village类:继承自land,用于配置游戏初始状态,包括地图、agents、camera等信息。
3.后端框架
后端代码在wounderland目录下,是一个独立的python项目,可以直接python setup.py install进行安装。这种设计主要是为了前后端分离,从而可以分别维护。wounderland中的模块基本按照agent常用架构设计,结构如下:
由game控制所有agent并与playground前端进行交互;timer平行于game进行时间管理;maze用来管理地图数据;llm model是llm模型服务的类,可以兼容各个厂商提供的llm service,此处感谢datawhale的教程,llm相关配置部分基本都来自于此:
项目核心模块即agent,其中各个部分作用为:
3.1 spatial:空间记忆
每个agent初始状态会有空间记忆,用来描述自己印象中去过的地方,空间信息在决策行动地点的时候会用到。随着游戏进行,每次agent会将自己感知到的新地点加入空间记忆。空间记忆以层次结构存储,如:
"the Ville": {
"artist's co-living space": {
"Francisco Lopez's room": [
"shelf",
"closet",
"bed",
"easel"
],
"Francisco Lopez's bathroom": [
"shower",
"bathroom sink",
"toilet"
],
"kitchen": [
"refrigerator",
"toaster",
"kitchen sink",
"cooking area"
],
"common room": [
"common room table",
"common room sofa"
]
},
...
}
3.2 schedule:日程安排(短期记忆)
agent初始状态没有日程安排,第一次思考时会根据agent个人的特点和时间生成schedule,生成方式为通过提示词,生成的日程安排是planning的基础(代码设计中,细分日程是每个小时开始时生成的,而小时日程是当天第一次规划的时候生成的):
schedule:
00:00~01:00: sleeping
01:00~02:00: sleeping
02:00~03:00: sleeping
03:00~04:00: sleeping
04:00~05:00: sleeping
05:00~06:00: sleeping
06:00~07:00: waking up and starting her morning routine
07:00~08:00: eating breakfast
08:00~09:00: preparing for the day's work at Hobbs Cafe
09:00~10:00: opening Hobbs Cafe and standing at the counter:
09:00~09:05: unlocking the doors and turning on the lights
09:05~09:15: checking the inventory and restocking items
09:15~09:25: brewing coffee and preparing the pastry display
09:25~09:30: setting up the outdoor seating area
09:30~09:40: greeting and seating customers
09:40~09:50: taking orders and serving customers
09:50~09:55: cleaning the counter and wiping down tables
09:55~10:00: restocking the condiments and utensils
10:00~11:00: continuing to serve customers at Hobbs Cafe
11:00~12:00: still at Hobbs Cafe, engaging with customers
12:00~13:00: taking a short break while still being present at Hobbs Cafe
13:00~14:00: back at the counter, serving customers
14:00~15:00: managing the cafe, ensuring everything is running smoothly
15:00~16:00: interacting with customers and promoting the upcoming Valentine's Day party
16:00~17:00: preparing for the closing routine, starting with tidying up the cafe
17:00~18:00: closing Hobbs Cafe and completing the closing routine
18:00~19:00: returning home and relaxing
19:00~20:00: enjoying her evening, perhaps watching TV or reading
20:00~21:00: getting ready for bed
21:00~22:00: winding down with a warm bath or a cup of tea
22:00~23:00: in bed, reading or listening to soft music
23:00~00:00: sleeping
3.3 associate:长期记忆
用于记录agent的所见所思所想以及与别的agent聊天的内容,agent相关行动都可能产生记忆,这些记忆被记录在associate中,在reflect和reaction时被取出进行总结加工。记忆分为三类:思考记忆,事件记忆和聊天记忆
agent在定制完schedule后会将schedule作为一条思考记忆保存到associate中,思考记忆信息如:
This is Isabella Rodriguez's plan for Monday February 13: wake up and complete the morning routine at 6:00 AM; eat breakfast at 7:00 AM; prepare for the day's work at Hobbs Cafe from 7:30 AM to 8:00 AM; open Hobbs Cafe at 8:00 AM and stand at the counter until 8:00 PM; close Hobbs Cafe at 8:00 PM and complete the closing routine; return home and relax from 9:00 PM to 11:00 PM; go to bed at 11:00 PM
agent思考过程会感知环境,此时会产生一些事件记忆例如:
idle_0:
event(P.1): bed <is> idle(bed is idle) @ the Ville:Isabella Rodriguez's apartment:main room:bed
duration: 20230213-09:30 ~ 20230315-09:30 (access: 20230213-09:30)
idle_2:
event(P.1): desk <is> idle(desk is idle) @ the Ville:Isabella Rodriguez's apartment:main room:desk
duration: 20230213-09:30 ~ 20230315-09:30 (access: 20230213-09:30)
idle_3:
event(P.1): refrigerator <is> idle(refrigerator is idle) @ the Ville:Isabella Rodriguez's apartment:main room:refrigerator
duration: 20230213-09:30 ~ 20230315-09:30 (access: 20230213-09:30)
idle_4:
event(P.1): closet <is> idle(closet is idle) @ the Ville:Isabella Rodriguez's apartment:main room:closet
duration: 20230213-09:30 ~ 20230315-09:30 (access: 20230213-09:30)
idle_5:
event(P.1): shelf <is> idle(shelf is idle) @ the Ville:Isabella Rodriguez's apartment:main room:shelf
duration: 20230213-09:30 ~ 20230315-09:30 (access: 20230213-09:30)
经过一些时间的演进,当两个agent距离较近的的时候可以感知到对方,此时agent会决定是否要和对方聊天,如果决定聊天,则产生聊天信息的记忆(summary部分内容)如:
Maria Lopez: Hey Isabella, I'm really sorry to bother you while you're working, but I just wanted to say that I've been coming here a lot lately and I really appreciate the atmosphere you've created here. It's a great place to study and relax.
Isabella Rodriguez: Thank you so much, Maria. I'm really glad you enjoy the atmosphere here. It's my passion to create a space where people can feel comfortable and at ease. Would you like to stay for our Valentine's Day party tomorrow? It's going to be a lot of fun, and I think it would be a great way to celebrate the spirit of the season with our regulars like you.
Maria Lopez: That sounds like a fantastic idea! I'd love to stay for the Valentine's Day party. Could you tell me more about it? What can I expect?
Isabella Rodriguez: The party will be a cozy affair with some light refreshments, live music, and a few games to get everyone in the mood for love. It's a great opportunity to meet new people or just enjoy the company of old friends. I hope you'll join us!
Maria Lopez: That sounds like a wonderful event, Isabella. I'm definitely looking forward to it. Thank you for the invitation!
Isabella Rodriguez: Great! I'm really happy to hear that you're excited about it. We'll have some delicious treats, like heart-shaped cookies and a special Valentine's Day menu. And the live music will be provided by a local acoustic duo, they're really talented and I think you'll enjoy their performance. It's going to be a night filled with love and good vibes.
Maria Lopez: That sounds like an amazing event, Isabella. I'm really looking forward to it. Do you need me to RSVP, or is it just a drop-in kind of thing? And are there any special instructions or dress code I should be aware of?
Isabella Rodriguez: The party is open to all our regulars and friends, so there's no need to RSVP. Just drop by on February 14th, from 5pm to 7pm, and enjoy the evening with us! And as for the dress code, it's casual and cozy – just come as you are. I want everyone to feel comfortable and relaxed. I'm really looking forward to seeing you there, Maria!
Maria Lopez: That sounds like a fantastic evening, Isabella. I'm really looking forward to it. And I appreciate the casual dress code – it's perfect for a cozy night like that. See you then!
Isabella Rodriguez: Sounds great, Maria! I'm really looking forward to seeing you here. And don't worry, you can just drop in – no need for formalities. It's going to be a night filled with love and good vibes, and I'm so happy to have you as part of our Hobbs Cafe community. See you tomorrow!
Maria Lopez: That's wonderful, Isabella. I'm really looking forward to the Valentine's Day party. I'll definitely be there! Thank you again for creating such a wonderful space and for the invitation.
Isabella Rodriguez: Maria, it's going to be a wonderful evening filled with love and great music. I can't wait to see you here! And remember, no need to dress up – just come as you are. Thank you for being a part of the Hobbs Cafe community. See you tomorrow!
summary:
Maria Lopez expresses gratitude to Isabella Rodriguez for the inviting atmosphere at Hobbs Cafe and accepts the invitation to the Valentine's Day party, which Isabella describes as a casual event with light refreshments, live music, and a festive spirit.
3.4 event && action:事件和行动
event是associate与action的基础,可以理解为事件的表达,采用经典的三元素:主<谓>宾,例如:
Isabella Rodriguez <is greeting and seating> customers(Isabella Rodriguez is greeting and seating customers)[ ] @ the Ville:Hobbs Cafe:cafe:cafe customer seating
同时为了更好的描述event,用describe来记录事件具体描述,emoji用来记录表情,address用于记录事件发生的地点。最终的event格式为:
主语<谓语>宾语(描述)[emoji] @ 事发地点
action一般包含两个事件(聊天类型的action除外),角色时间和物品事件,一般角色事件的宾语就是物品事件的主语(但也有例外,物品事件完全由llm提取生成,不一定准确),前者描述了agent要做什么,后者描述物品状态。角色事件分为action和chat两种类型,分别描述行动和聊天,例如:
行动action
status: doing[20230213-09:30~20230213-09:40]
event: Isabella Rodriguez <is greeting and seating> customers(Isabella Rodriguez is greeting and seating customers)[ ] @ the Ville:Hobbs Cafe:cafe:cafe customer seating
object: Isabella Rodriguez <occupies> cafe customer seating(cafe customer seating is occupied by Isabella Rodriguez as she greets and seats customers)[ ] @ the Ville:Hobbs Cafe:cafe:cafe customer seating
聊天action
status: doing[20230213-09:40~20230213-09:51]
event: Maria Lopez <chat with> Isabella Rodriguez(Maria Lopez expresses gratitude to Isabella Rodriguez for the inviting atmosphere at Hobbs Cafe and accepts the invitation to the Valentine's Day party, which Isabella describes as a casual event with light refreshments, live music, and a festive spirit)[ ] @ <persona>:Isabella Rodriguezf
3.5 path:行动路线
根据action中主角事件的address定位目的地,在通过动态规划找到前往目的地的最佳路径,最终就表现为agent在地图上行走,path记录为一系列的tile坐标,起点是agent当前坐标,重点是action的发生地点,如:
tile:
coord[77,19]: the Ville:Hobbs Cafe:cafe:kitchen sink
events:
e_0: kitchen sink <is> idle @ the Ville:Hobbs Cafe:cafe:kitchen sink
e_1: Isabella Rodriguez <is engaging with> customers(Isabella Rodriguez is still at Hobbs Cafe, engaging with customers (checking on the status of food orders with the kitchen))
action:
status: doing[20230213-11:40~20230213-11:45]
event: Isabella Rodriguez <is> still at Hobbs Cafe, engaging with customers (handling payment for a table of customers)(Isabella Rodriguez is still at Hobbs Cafe, engaging with customers (handling payment for a table of customers))[ ] @ the Ville:The Rose and Crown Pub:pub:cooking area
object: cooking area <is> idle(cooking area is idle)[ ] @ the Ville:The Rose and Crown Pub:pub:cooking area
path: 76,19-75,19-75,20-75,21-75,22-75,23-75,24-75,25-75,26-76,26-76,27-76,28-76,29-76,30-75,30-74,30-73,30-72,30-71,30-70,30-69,30-68,30-67,30-66,30-65,30-64,30-63,30-62,30-61,30-60,30-59,30-58,30-57,30-57,29-57,28-57,27-57,26-57,25-57,24-57,23-56,23-55,23-54,23-53,23-53,22-53,21-53,20-54,20-55,20-56,20-57,20-58,20-59,20-60,20-61,20
3.6 scratch:提示词模块
scratch模块是整个agent的灵魂,其本质就是提示词工程。根据每个agent的个人特性和任务生成提示词,调用llm然后提取关键信息,一个scratch包含三个部分:提示词promt,llm服务的返回response以及后处理后得到的结果output,例如:
-------------------- Isabella Rodriguez.wake_up --------------------
<PROMPT>:
"""
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on *February 14th, 2023, from 5pm to 7pm*.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and stands at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
In general, Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Isabella Rodriguez's wake up hour:
"""
Output the response to the prompt above in hour. The output should ONLY contain ONE hour value.
Example output hour:
8:00 am
<RESPONSE[0/1]>:
Isabella Rodriguez's wake up hour is:
6:00 am
<OUTPUT>:
6
llm返回的结果稳定性不强,不知道是不是和白嫖的服务能力上限不高有关,好在通过精心设计的prompt和正则表达式的奇技淫巧,还是可以表稳定的获得正确的输出。
4.Agent思考流程
整个小镇运作核心就是提示词工程,所有agent的思考行动都是通过提示词工程得到结果,下面以一次思考过程(Agent.think)为例整体展示一下
4.1 制定日程(make_schedule)
agent的一天都从制定日程开始,日程在一天中只会制定一次,后续行动都是按照日程进行,只有在决定chat的时候才会更改已有日程。首先生成起床时间,见3.6的example,然后分三步进行日程规划:
4.1.1 生成粗略日程
<PROMPT>:
"""
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on *February 14th, 2023, from 5pm to 7pm*.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and stands at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
In general, Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Today is Monday February 13. Here is Isabella Rodriguez's plan today in broad-strokes (with the time of the day. e.g., eat breakfast at 7:00 AM, have a lunch at 12:00 PM, watch TV at 7:00 PM):
1) wake up and complete the morning routine at 6:00 AM.
2)
"""
<RESPONSE[0/1]>:
1) wake up and complete the morning routine at 6:00 AM.
2) eat breakfast at 7:00 AM.
3) prepare for the day's work at Hobbs Cafe from 7:30 AM to 8:00 AM.
4) open Hobbs Cafe at 8:00 AM and stand at the counter until 8:00 PM.
5) close Hobbs Cafe at 8:00 PM and complete the closing routine.
6) return home and relax from 9:00 PM to 11:00 PM.
7) go to bed at 11:00 PM.
生成的粗略日程会被当作agent当天的第一条记忆加入associate中,注意所有加入associate中的记忆都会被评估重要程度(poignacy)并提升agent的poignacy数值,数值达到一定程度后触发反思。
如果agent在开始游戏的时间点已经有一些记忆,会对记忆进行追溯,提取出比较关键的信息总结出agent的currently信息,此信息在后续多处决策过程中都会起作用:
首先回溯并提取对plan有影响的记忆
<PROMPT>:
[Statements]
Monday February 13 -- 09:30 AM: This is Isabella Rodriguez's plan for Monday February 13: wake up and complete the morning routine at 6:00 AM; eat breakfast at 7:00 AM; prepare for the day at Hobbs Cafe from 7:30 AM to 8:00 AM; open Hobbs Cafe at 8:00 AM and stand at the counter until 8:00 PM; close Hobbs Cafe at 8:00 PM and complete end-of-day tasks until 9:00 PM; return home and relax from 9:00 PM to 11:00 PM; go to bed at 11:00 PM
Monday February 13 -- 10:50 AM: Isabella Rodriguez is engaging in friendly conversation with customers
Monday February 13 -- 10:30 AM: Isabella Rodriguez is making coffee and preparing drinks for customers
Monday February 13 -- 10:50 AM: cafe customer seating is occupied by Isabella Rodriguez and other customers, and is being used for friendly conversation
Monday February 13 -- 10:10 AM: behind the cafe counter is likely to be in a state of preparation or service, with staff members like Isabella Rodriguez attending to customers, taking orders, and preparing drinks and food. The counter itself might have a cash register, menus, and various tools and ingredients needed for making beverages and snacks. There could be a display of pastries or drinks ready for sale, and the area might be bustling with activity as customers place their orders and staff members fulfill them
Monday February 13 -- 10:00 AM: Isabella Rodriguez is opening Hobbs Cafe and standing at the counter (greeting and welcoming the first customers of the day)
Monday February 13 -- 10:40 AM: Isabella Rodriguez is delivering drinks and snacks to customers' tables
Monday February 13 -- 10:50 AM: Maria Lopez is browsing through the latest posts in her favorite online communities
Monday February 13 -- 09:40 AM: Isabella Rodriguez is setting up the cash register and ensuring the POS system is functioning
Monday February 13 -- 09:50 AM: Maria Lopez is taking notes on key concepts and formulas
Monday February 13 -- 10:20 AM: Isabella Rodriguez is taking orders from customers at the counter
Monday February 13 -- 09:40 AM: cooking area is being used for food preparation and service, with the cash register and POS system in use, indicating that the area is active and operational
Monday February 13 -- 10:30 AM: Maria Lopez is checking her phone for messages and notifications
Monday February 13 -- 10:40 AM: cafe customer seating is occupied by Maria Lopez who is using it to update her social media status
Monday February 13 -- 10:10 AM: Maria Lopez is continuing to study physics at Hobbs Cafe (resuming practice problems and then discussing physics concepts with a study partner)
Monday February 13 -- 10:20 AM: Maria Lopez is ordering a snack and a drink from the cafe
Monday February 13 -- 09:50 AM: kitchen sink is running with water
Monday February 13 -- 10:50 AM: piano is being used as a table or surface for browsing the internet
Monday February 13 -- 10:20 AM: behind the cafe counter is idle
Given the statements above, is there anything that Isabella Rodriguez should remember when planing for Tuesday February 14?
If there is any scheduling information, be as specific as possible (include date, time, and location if stated in the statement)
Write the response from Isabella Rodriguez's perspective in lines, each line contains ONE thing to remember.
<RESPONSE[0/1]>:
1. Remember to set up the cash register and ensure the POS system is functioning before opening Hobbs Cafe.
2. Keep the kitchen sink clean and ready for use during the day.
3. Engage in friendly conversation with customers to create a welcoming atmosphere.
4. Use the piano as a decorative piece, not as a table or surface for browsing the internet.
5. Ensure the cooking area is well-prepared for food service and that all tools and ingredients are in place.
6. Schedule time to update social media and browse online communities after the cafe has closed.
7. Plan to take orders from customers at the counter during peak hours.
8. Prepare for the day at Hobbs Cafe from 7:30 AM to 8:00 AM to ensure a smooth opening.
9. Stand at the counter from 8:00 AM to 8:00 PM to greet customers and manage the flow of service.
10. Close Hobbs Cafe at 8:00 PM and complete end-of-day tasks until 9:00 PM.
11. Relax at home from 9:00 PM to 11:00 PM to unwind after a long day.
12. Go to bed at 11:00 PM to ensure a good night's rest for the next day.
然后提取对思考重要的记忆
<PROMPT>:
[Statements]
Monday February 13 -- 09:30 AM: This is Isabella Rodriguez's plan for Monday February 13: wake up and complete the morning routine at 6:00 AM; eat breakfast at 7:00 AM; prepare for the day at Hobbs Cafe from 7:30 AM to 8:00 AM; open Hobbs Cafe at 8:00 AM and stand at the counter until 8:00 PM; close Hobbs Cafe at 8:00 PM and complete end-of-day tasks until 9:00 PM; return home and relax from 9:00 PM to 11:00 PM; go to bed at 11:00 PM
Monday February 13 -- 10:50 AM: Isabella Rodriguez is engaging in friendly conversation with customers
Monday February 13 -- 10:30 AM: Isabella Rodriguez is making coffee and preparing drinks for customers
Monday February 13 -- 10:50 AM: cafe customer seating is occupied by Isabella Rodriguez and other customers, and is being used for friendly conversation
Monday February 13 -- 10:10 AM: behind the cafe counter is likely to be in a state of preparation or service, with staff members like Isabella Rodriguez attending to customers, taking orders, and preparing drinks and food. The counter itself might have a cash register, menus, and various tools and ingredients needed for making beverages and snacks. There could be a display of pastries or drinks ready for sale, and the area might be bustling with activity as customers place their orders and staff members fulfill them
Monday February 13 -- 10:00 AM: Isabella Rodriguez is opening Hobbs Cafe and standing at the counter (greeting and welcoming the first customers of the day)
Monday February 13 -- 10:40 AM: Isabella Rodriguez is delivering drinks and snacks to customers' tables
Monday February 13 -- 10:50 AM: Maria Lopez is browsing through the latest posts in her favorite online communities
Monday February 13 -- 09:40 AM: Isabella Rodriguez is setting up the cash register and ensuring the POS system is functioning
Monday February 13 -- 09:50 AM: Maria Lopez is taking notes on key concepts and formulas
Monday February 13 -- 10:20 AM: Isabella Rodriguez is taking orders from customers at the counter
Monday February 13 -- 09:40 AM: cooking area is being used for food preparation and service, with the cash register and POS system in use, indicating that the area is active and operational
Monday February 13 -- 10:30 AM: Maria Lopez is checking her phone for messages and notifications
Monday February 13 -- 10:40 AM: cafe customer seating is occupied by Maria Lopez who is using it to update her social media status
Monday February 13 -- 10:10 AM: Maria Lopez is continuing to study physics at Hobbs Cafe (resuming practice problems and then discussing physics concepts with a study partner)
Monday February 13 -- 10:20 AM: Maria Lopez is ordering a snack and a drink from the cafe
Monday February 13 -- 09:50 AM: kitchen sink is running with water
Monday February 13 -- 10:50 AM: piano is being used as a table or surface for browsing the internet
Monday February 13 -- 10:20 AM: behind the cafe counter is idle
Given the statements above, how might we summarize Isabella Rodriguez's feelings up to now?
Write the response from Isabella Rodriguez's perspective in ONE sentence.
<RESPONSE[0/1]>:
Isabella Rodriguez feels content and engaged as she interacts with customers and manages the daily operations of Hobbs Cafe, enjoying the routine and the social aspect of her work.
最后将比较重要的记忆汇总得到新的currently(当前人物描述):
<PROMPT>:
Isabella Rodriguez's status from Monday February 13:
Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on *February 14th, 2023, from 5pm to 7pm*.
Isabella Rodriguez remember these things at the end of Monday February 13:
Remember to set up the cash register and ensure the POS system is functioning before opening Hobbs Cafe. Keep the kitchen sink clean and ready for use during the day. Engage in friendly conversation with customers to create a welcoming atmosphere. Use the piano as a decorative piece, not as a table or surface for browsing the internet. Ensure the cooking area is well-prepared for food service and that all tools and ingredients are in place. Schedule time to update social media and browse online communities after the cafe has closed. Plan to take orders from customers at the counter during peak hours. Prepare for the day at Hobbs Cafe from 7:30 AM to 8:00 AM to ensure a smooth opening. Stand at the counter from 8:00 AM to 8:00 PM to greet customers and manage the flow of service. Close Hobbs Cafe at 8:00 PM and complete end-of-day tasks until 9:00 PM. Relax at home from 9:00 PM to 11:00 PM to unwind after a long day. Go to bed at 11:00 PM to ensure a good night's rest for the next day
Isabella Rodriguez's feeling at the end of Monday February 13:
Isabella Rodriguez feels content and engaged as she interacts with customers and manages the daily operations of Hobbs Cafe, enjoying the routine and the social aspect of her work.
It is now Tuesday February 14. Given the above, write Isabella Rodriguez's status for Tuesday February 14 that reflects Isabella Rodriguez's thoughts at the end of Monday February 13.
Write this in third-person talking about Isabella Rodriguez in ONE sentence. Follow this format below:
Status: <new status>
<RESPONSE[0/1]>:
Status: Isabella Rodriguez is looking forward to hosting her Valentine's Day party at Hobbs Cafe, having prepared the cafe and herself for a day of celebration and customer service.
4.1.2 生成小时日程
<PROMPT>:
Hourly schedule format:
[0:00 AM] Activity: [Fill in]
[1:00 AM] Activity: [Fill in]
[2:00 AM] Activity: [Fill in]
[3:00 AM] Activity: [Fill in]
[4:00 AM] Activity: [Fill in]
[5:00 AM] Activity: [Fill in]
[6:00 AM] Activity: [Fill in]
[7:00 AM] Activity: [Fill in]
[8:00 AM] Activity: [Fill in]
[9:00 AM] Activity: [Fill in]
[10:00 AM] Activity: [Fill in]
[11:00 AM] Activity: [Fill in]
[0:00 PM] Activity: [Fill in]
[1:00 PM] Activity: [Fill in]
[2:00 PM] Activity: [Fill in]
[3:00 PM] Activity: [Fill in]
[4:00 PM] Activity: [Fill in]
[5:00 PM] Activity: [Fill in]
[6:00 PM] Activity: [Fill in]
[7:00 PM] Activity: [Fill in]
[8:00 PM] Activity: [Fill in]
[9:00 PM] Activity: [Fill in]
[10:00 PM] Activity: [Fill in]
[11:00 PM] Activity: [Fill in]
========
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on *February 14th, 2023, from 5pm to 7pm*.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and stands at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
Here the originally intended hourly breakdown of Isabella Rodriguez's schedule today:
wake up and complete the morning routine at 6:00 AM; eat breakfast at 7:00 AM; prepare for the day's work at Hobbs Cafe from 7:30 AM to 8:00 AM; open Hobbs Cafe at 8:00 AM and stand at the counter until 8:00 PM; close Hobbs Cafe at 8:00 PM and complete the closing routine; return home and relax from 9:00 PM to 11:00 PM; go to bed at 11:00 PM
[0:00 AM] Activity: Isabella Rodriguez is sleeping
[1:00 AM] Activity: Isabella Rodriguez is sleeping
[2:00 AM] Activity: Isabella Rodriguez is sleeping
[3:00 AM] Activity: Isabella Rodriguez is sleeping
[4:00 AM] Activity: Isabella Rodriguez is sleeping
[5:00 AM] Activity: Isabella Rodriguez is sleeping
[6:00 AM] Activity: Isabella Rodriguez is
<RESPONSE[0/1]>:
[0:00 AM] Activity: Isabella Rodriguez is sleeping
[1:00 AM] Activity: Isabella Rodriguez is sleeping
[2:00 AM] Activity: Isabella Rodriguez is sleeping
[3:00 AM] Activity: Isabella Rodriguez is sleeping
[4:00 AM] Activity: Isabella Rodriguez is sleeping
[5:00 AM] Activity: Isabella Rodriguez is sleeping
[6:00 AM] Activity: Isabella Rodriguez is waking up and starting her morning routine
[7:00 AM] Activity: Isabella Rodriguez is eating breakfast
[8:00 AM] Activity: Isabella Rodriguez is preparing for the day's work at Hobbs Cafe
[9:00 AM] Activity: Isabella Rodriguez is opening Hobbs Cafe and standing at the counter
[10:00 AM] Activity: Isabella Rodriguez is continuing to serve customers at Hobbs Cafe
[11:00 AM] Activity: Isabella Rodriguez is still at Hobbs Cafe, engaging with customers
[0:00 PM] Activity: Isabella Rodriguez is taking a short break while still being present at Hobbs Cafe
[1:00 PM] Activity: Isabella Rodriguez is back at the counter, serving customers
[2:00 PM] Activity: Isabella Rodriguez is managing the cafe, ensuring everything is running smoothly
[3:00 PM] Activity: Isabella Rodriguez is interacting with customers and promoting the upcoming Valentine's Day party
[4:00 PM] Activity: Isabella Rodriguez is preparing for the closing routine, starting with tidying up the cafe
[5:00 PM] Activity: Isabella Rodriguez is closing Hobbs Cafe and completing the closing routine
[6:00 PM] Activity: Isabella Rodriguez is returning home and relaxing
[7:00 PM] Activity: Isabella Rodriguez is enjoying her evening, perhaps watching TV or reading
[8:00 PM] Activity: Isabella Rodriguez is getting ready for bed
[9:00 PM] Activity: Isabella Rodriguez is winding down with a warm bath or a cup of tea
[10:00 PM] Activity: Isabella Rodriguez is in bed, reading or listening to soft music
[11:00 PM] Activity: Isabella Rodriguez is sleeping
4.1.3 按照当前时间生成细分日程
<PROMPT>:
Describe subtasks in 5 min increments.
[Example]
Name: Kelly Bronson
Age: 35
Backstory: Kelly always wanted to be a teacher, and now she teaches kindergarten. During the week, she dedicates herself to her students, but on the weekends, she likes to try out new restaurants and hang out with friends. She is very warm and friendly, and loves caring for others.
Personality: sweet, gentle, meticulous
Location: Kelly is in an older condo that has the following areas: {kitchen, bedroom, dining, porch, office, bathroom, living room, hallway}.
Currently: Kelly is a teacher during the school year. She teaches at the school but works on lesson plans at home. She is currently living alone in a single bedroom condo.
Daily plan requirement: Kelly is planning to teach during the morning and work from home in the afternoon.s
Today is Saturday May 10. From 08:00AM ~ 09:00AM, Kelly is planning on having breakfast, from 09:00AM ~ 10:00PM, Kelly is planning on working on the next day's kindergarten lesson plan, and from 10:00AM ~ 13PM, Kelly is planning on taking a break.
In 5 min increments, list the subtasks Kelly does when Kelly is working on the next day's kindergarten lesson plan from 09:00am ~ 10:00pm (total duration in minutes: 60):
<subtasks>:
1) Kelly <is> reviewing the kindergarten curriculum standards. (duration: 15, left: 45)
2) Kelly <is> brainstorming ideas for the lesson. (duration: 10, left: 35)
3) Kelly <is> creating the lesson plan. (duration: 10, left: 25)
8) Kelly <is> printing the lesson plan. (duration: 10, left: 15)
9) Kelly <is> putting the lesson plan in her bag. (duration: 15, left: 0)
Given example above, please list the subtasks of the following task.
[TASK]
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on *February 14th, 2023, from 5pm to 7pm*.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and stands at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
Today is Monday February 13. From 08:00AM ~ 09:00AM, Isabella Rodriguez is planning on preparing for the day's work at Hobbs Cafe, 09:00AM ~ 10:00AM, Isabella Rodriguez is planning on opening Hobbs Cafe and standing at the counter, 10:00AM ~ 11:00AM, Isabella Rodriguez is planning on continuing to serve customers at Hobbs Cafe
In 5 min increments, list the subtasks Isabella Rodriguez does when Isabella Rodriguez is opening Hobbs Cafe and standing at the counter from 09:00AM ~ 10:00AM (total duration in minutes 60):
<subtasks>:
1) Isabella Rodriguez <is>
<RESPONSE[0/1]>:
<subtasks>:
1) Isabella Rodriguez <is> unlocking the doors and turning on the lights. (duration: 5, left: 55)
2) Isabella Rodriguez <is> checking the inventory and restocking items. (duration: 10, left: 45)
3) Isabella Rodriguez <is> brewing coffee and preparing the pastry display. (duration: 10, left: 35)
4) Isabella Rodriguez <is> setting up the outdoor seating area. (duration: 5, left: 30)
5) Isabella Rodriguez <is> greeting and seating customers. (duration: 10, left: 20)
6) Isabella Rodriguez <is> taking orders and serving customers. (duration: 10, left: 10)
7) Isabella Rodriguez <is> cleaning the counter and wiping down tables. (duration: 5, left: 5)
8) Isabella Rodriguez <is> restocking the condiments and utensils. (duration: 5, left: 0)
4.2 感知环境(percept)
agent行动前会先对环境进行感知,也就是对自己视野范围内的tile上面存在的event进行记录,这里有个槽点,似乎agent隔着墙也可以感受到墙后面的事件-_-...可以作为改良的一个点,就是根据tile是否为墙(collision=True)来判断某些tile是否能被感知。
agent对于有意义的事件(宾语不是idle)会判断一下重要程度(重要程度会被累计,到一定程度后触发反思):
<PROMPT>:
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on *February 14th, 2023, from 5pm to 7pm*.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and stands at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
On the scale of 1 to 10, where 1 is purely mundane (e.g., brushing teeth, making bed) and 10 is extremely poignant (e.g., a break up, college acceptance), rate the likely poignancy of the following event for Isabella Rodriguez.
Each event should ONLY be rate with ONE integer on the scale of 1 to 10.
-----
Event: brushing teeth. Rate: 1
-----
Event: making bed. Rate: 1
-----
Event: a break up. Rate: 10
-----
Event: college acceptance. Rate: 10
-----
Event: cafe customer seating is occupied by Isabella Rodriguez as she greets and seats customers. Rate:
<RESPONSE[0/1]>:
Event: cafe customer seating is occupied by Isabella Rodriguez as she greets and seats customers. Rate: 5
最终汇总所有感知到的事件,注意这里只会将跟自己无关(主语不是自己)的事件作为planning的依据:
idle_0:
event(P.1): cafe customer seating <is> idle(cafe customer seating is idle) @ the Ville:Hobbs Cafe:cafe:cafe customer seating
duration: 20230213-09:40 ~ 20230315-09:40 (access: 20230213-09:40)
idle_2:
event(P.1): cooking area <is> idle(cooking area is idle) @ the Ville:Hobbs Cafe:cafe:cooking area
duration: 20230213-09:40 ~ 20230315-09:40 (access: 20230213-09:40)
idle_3:
event(P.1): kitchen sink <is> idle(kitchen sink is idle) @ the Ville:Hobbs Cafe:cafe:kitchen sink
duration: 20230213-09:40 ~ 20230315-09:40 (access: 20230213-09:40)
idle_4:
event(P.1): behind the cafe counter <is> idle(behind the cafe counter is idle) @ the Ville:Hobbs Cafe:cafe:behind the cafe counter
duration: 20230213-09:40 ~ 20230315-09:40 (access: 20230213-09:40)
idle_5:
event(P.1): refrigerator <is> idle(refrigerator is idle) @ the Ville:Hobbs Cafe:cafe:refrigerator
duration: 20230213-09:40 ~ 20230315-09:40 (access: 20230213-09:40)
idle_6:
event(P.1): piano <is> idle(piano is idle) @ the Ville:Hobbs Cafe:cafe:piano
duration: 20230213-09:40 ~ 20230315-09:40 (access: 20230213-09:40)
4.3 制定计划(make_plan)
首先根据当前的细分schedule判断agent是否是sleeping状态,如果是就不制定计划,并且不会寻路(老老实实躺在床上就好);如果不符合,则开始制定计划。
4.3.1 反应环境(reaction)
根据感知到的信息定位出关键信息,也就是包含其他角色或者宾语不为idle的事件,然后对这一事件进行决策,是否进行chat以及是否等待对方完成计划(触发概率较小,暂不考虑):
<PROMPT>:
Task -- given context, determine whether the subject will initiate a conversation with another.
Context:
This is Isabella Rodriguez's plan for Monday February 13: wake up and complete the morning routine at 6:00 AM; eat breakfast at 7:00 AM; prepare for the day's work at Hobbs Cafe from 7:30 AM to 8:00 AM; open Hobbs Cafe at 8:00 AM and work at the counter until 8:00 PM; close Hobbs Cafe at 8:00 PM and complete the closing routine; return home and relax from 9:00 PM to 11:00 PM; go to bed at 11:00 PM
Right now, it is February 13, 2023, 10:20:40 AM.
Isabella Rodriguez is already take orders customers
Maria Lopez is already order breakfast
Question: Would Isabella Rodriguez initiate a conversation with Maria Lopez?
Reasoning: Let's think step by step.
Answer in "yes" or "no":
<RESPONSE[0/1]>:
Yes
如果决定要交谈,那么首先要确定一下两人的关系(试了一下其实这个relation的结果对聊天内容影响很小):
<PROMPT>:
"""
[Statements]
0. Maria Lopez is discussing difficult concepts with a study group via video call
1. behind the cafe counter is likely to be in a state of preparation or service, with staff members like Isabella Rodriguez attending to customers, taking orders, and preparing drinks and food. The counter itself might have a cash register open, menus on display, and perhaps some ingredients or tools of the trade like coffee makers, measuring cups, and spoons. The atmosphere would be bustling with activity as the staff works to provide service to the customers
2. This is Maria Lopez's plan for Monday February 13: Wake up and complete the morning routine at 6:00 AM; Eat breakfast at 7:00 AM; Study physics at Hobbs Cafe from 8:00 AM to 12:00 PM; Have lunch at 1:00 PM; Stream on Twitch from 2:00 PM to 8:00 PM; Dinner at 7:00 PM; Relax and watch TV from 8:00 PM to 10:00 PM; Prepare for bed and sleep at 11:00 PM
3. Maria Lopez is taking notes on key concepts
4. Maria Lopez is solving practice problems from the textbook
5. library table is being used for studying/working on practice problems
6. cafe customer seating is available/unoccupied, as Maria Lopez is walking to the cafe, indicating that she has not yet taken a seat. Once she sits down, the seating would be considered occupied
7. library table is being used for taking notes on key concepts
8. common room table is being used as a work surface for a video call
Based on the statements above, summarize Maria Lopez and Isabella Rodriguez's relationship (e.g., Tom and Jeo are friends, Elin and John are playing games). What do they feel or know about each other?
"""
Output the response to the prompt above in sentence. The output should be ONE sentence that describe the relationship.
Example output sentence:
Jane and Tom are friends
<RESPONSE[0/1]>:
Maria Lopez and Isabella Rodriguez's relationship is not explicitly stated in the provided statements, but it can be inferred that they are likely study group members or colleagues who are engaged in a professional or academic setting.
然后开始在agent和聊天对象之间反复生成聊天内容,直到某一方决定终止聊天或者超过一定聊天轮数(大概是怕两个人一直聊下去...),这里使用了associate的检索(Here is the memory that is in XXX's head下面的内容就是检索出来的记忆),检索出来的内容无脑交给llm进行总结:
<PROMPT>:
Context for the task:
PART 1. Abstract of Maria Lopez
Here is a brief description of Maria Lopez:
Name: Maria Lopez
Age: 21
Innate traits: energetic, enthusiastic, inquisitive
Learned traits: Maria Lopez is a student at Oak Hill College studying physics and a part time Twitch game streamer who loves to connect with people and explore new ideas.
Currently: Maria Lopez is working on her physics degree and streaming games on Twitch to make some extra money. She visits Hobbs Cafe for studying and eating just about everyday.
Lifestyle: Maria Lopez goes to bed around midnight, awakes up around 10am, eats dinner around 7pm.
Daily plan requirement: Maria Lopez spends at least 6 hours a day Twitch streaming or gaming.
Current Date: Monday February 13
Here is the memory that is in Maria Lopez's head:
- Maria Lopez is discussing difficult concepts with a study group via video call
- Maria Lopez is taking notes on key concepts
- This is Maria Lopez's plan for Monday February 13: Wake up and complete the morning routine at 6:00 AM; Eat breakfast at 7:00 AM; Study physics at Hobbs Cafe from 8:00 AM to 12:00 PM; Have lunch at 1:00 PM; Stream on Twitch from 2:00 PM to 8:00 PM; Dinner at 7:00 PM; Relax and watch TV from 8:00 PM to 10:00 PM; Prepare for bed and sleep at 11:00 PM
- Maria Lopez is solving practice problems from the textbook
- behind the cafe counter is likely to be in a state of preparation or service, with staff members like Isabella Rodriguez attending to customers, taking orders, and preparing drinks and food. The counter itself might have a cash register open, menus on display, and perhaps some ingredients or tools of the trade like coffee makers, measuring cups, and spoons. The atmosphere would be bustling with activity as the staff works to provide service to the customers
- library table is being used for studying/working on practice problems
- cafe customer seating is available/unoccupied, as Maria Lopez is walking to the cafe, indicating that she has not yet taken a seat. Once she sits down, the seating would be considered occupied
PART 2. Past Context
Current Location: cafe customer seating in cafe
Current Context: Maria Lopez was is walking to Hobbs Cafe when Maria Lopez saw Isabella Rodriguez in the middle of is taking orders from customers at the counter.
Maria Lopez is initiating a conversation with Isabella Rodriguez.
Maria Lopez and Isabella Rodriguez are chatting. Here is their conversation so far:
[The conversation has not started yet -- start it!]
---
Task: Given the context above, what should Maria Lopez say to Isabella Rodriguez next in the conversation? And did it end the conversation?
Output a json of the following format:
{
"Maria Lopez": "<Maria Lopez's utterance>",
"End the conversation": "<json Boolean>"
}
<RESPONSE[0/1]>:
{
"Maria Lopez": "Hey Isabella, how's your day going so far? I'm really excited about the study group tonight. Do you think you'll be able to make it?",
"End the conversation": false
}
-----------------------
<PROMPT>:
Context for the task:
PART 1. Abstract of Isabella Rodriguez
Here is a brief description of Isabella Rodriguez:
Name: Isabella Rodriguez
Age: 34
Innate traits: friendly, outgoing, hospitable
Learned traits: Isabella Rodriguez is a cafe owner of Hobbs Cafe who loves to make people feel welcome. She is always looking for ways to make the cafe a place where people can come to relax and enjoy themselves.
Currently: Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on *February 14th, 2023, from 5pm to 7pm*.
Lifestyle: Isabella Rodriguez goes to bed around 11pm, awakes up around 6am.
Daily plan requirement: Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and stands at the counter until 8pm, at which point she closes the cafe.
Current Date: Monday February 13
Here is the memory that is in Isabella Rodriguez's head:
- Isabella Rodriguez is taking orders and serving customers
- Isabella Rodriguez is greeting and seating customers
- behind the cafe counter is likely to be in a state of preparation or service, with staff members like Isabella Rodriguez attending to customers, taking orders, and preparing drinks and food. The counter itself might have a cash register open, menus on display, and perhaps some ingredients or tools of the trade like coffee makers, measuring cups, and spoons. The atmosphere would be bustling with activity as the staff works to provide service to the customers
- This is Isabella Rodriguez's plan for Monday February 13: wake up and complete the morning routine at 6:00 AM; eat breakfast at 7:00 AM; prepare for the day at Hobbs Cafe from 7:30 AM to 8:00 AM; open Hobbs Cafe at 8:00 AM and stand at the counter until 8:00 PM; close Hobbs Cafe at 8:00 PM and complete end-of-day tasks until 9:00 PM; return home and relax from 9:00 PM to 11:00 PM; go to bed at 11:00 PM
- cafe customer seating is being used for customers to sit and enjoy their drinks and food
- Isabella Rodriguez is cleaning the counter and wiping down tables
- kitchen sink is running with water and being used for cleaning purposes
PART 2. Past Context
Current Location: behind the cafe counter in cafe
Current Context: Isabella Rodriguez was is taking orders from customers at the counter when Isabella Rodriguez saw Maria Lopez in the middle of is walking to Hobbs Cafe.
Isabella Rodriguez is initiating a conversation with Maria Lopez.
Isabella Rodriguez and Maria Lopez are chatting. Here is their conversation so far:
Maria Lopez: Hey Isabella, how's your day going so far? I'm really excited about the study group tonight. Do you think you'll be able to make it?
---
Task: Given the context above, what should Isabella Rodriguez say to Maria Lopez next in the conversation? And did it end the conversation?
Output a json of the following format:
{
"Isabella Rodriguez": "<Isabella Rodriguez's utterance>",
"End the conversation": "<json Boolean>"
}
<RESPONSE[0/1]>:
{
"Isabella Rodriguez": "Hey Maria, it's great to see you! My day has been busy but fulfilling. And yes, I'm really looking forward to the study group tonight. I think it's going to be a great opportunity to discuss our progress and learn from each other. See you there!",
"End the conversation": "false"
}
...
接下来对聊天内容进行汇总,总结出核心内容并加入associate中
<PROMPT>:
Conversation:
Maria Lopez: Hey Isabella, how's your day going so far? I'm really excited about the study group tonight. Do you think you'll be able to make it?
Isabella Rodriguez: Hey Maria, it's great to see you! My day has been busy but fulfilling. And yes, I'm really looking forward to the study group tonight. I think it's going to be a great opportunity to discuss our progress and learn from each other. See you there!
Maria Lopez: I'll see you at Hobbs Cafe around 6:30 PM. Let's grab a table and get ready for the study group!
Isabella Rodriguez: I'll make sure to clear a table for us so we can have a quiet spot to work. See you at 6:30 PM!
Maria Lopez: Great, thanks for doing that, Isabella. See you at 6:30 PM sharp! I'm really looking forward to diving into the material tonight.
Summarize the conversation above in one short sentence without comma:
<RESPONSE[0/1]>:
Maria Lopez and Isabella Rodriguez are excited about the study group tonight and plan to meet at Hobbs Cafe at 6:30 PM to discuss their progress and learn from each other.
接下来的操作还是比较有意思的,agent会根据聊天内容的总结重新生成当前时间段内的细分日程,也就是说聊天这种on-time的情况可以实时改变agent的计划,这也给整个游戏过程带来了一些非确定性:
<PROMPT>:
Here was Maria Lopez's originally planned schedule from 10:00 to 11:00.
[10:00 ~ 10:05] taking a break or having a snack (walking to Hobbs Cafe)
[10:05 ~ 10:10] taking a break or having a snack (ordering a snack and a drink)
[10:10 ~ 10:15] taking a break or having a snack (finding a comfortable seat to relax)
[10:15 ~ 10:25] taking a break or having a snack (checking her phone and responding to messages)
[10:25 ~ 10:35] taking a break or having a snack (reading through her physics notes)
[10:35 ~ 10:45] taking a break or having a snack (people-watching and observing the cafe atmosphere)
[10:45 ~ 10:55] taking a break or having a snack (enjoying her snack and drink)
[10:55 ~ 11:00] taking a break or having a snack (paying for her snack and drink)
But Maria Lopez unexpectedly end up with the following event for 2 minutes:
Maria Lopez and Isabella Rodriguez are excited about the study group tonight and plan to meet at Hobbs Cafe at 6:30 PM to discuss their progress and learn from each other.
Revise Maria Lopez's schedule from 10:00 to 11:00 accordingly (it has to end by 11:00).
The revised schedule:
[10:00 ~ 10:05] taking a break or having a snack (walking to Hobbs Cafe)
[10:05 ~ 10:10] taking a break or having a snack (ordering a snack and a drink)
[10:10 ~ 10:10] taking a break or having a snack (finding a comfortable seat to relax)
[10:10 ~ 10:12] taking a break or having a snack (and Isabella Rodriguez are excited about the study group tonight and plan to meet at Hobbs Cafe at 6:30 PM to discuss their progress and learn from each other)
[10:12 ~
<RESPONSE[0/1]>:
Given the unexpected event of Maria Lopez meeting Isabella Rodriguez to discuss the study group, we can adjust the schedule as follows:
[10:00 ~ 10:05] taking a break or having a snack (walking to Hobbs Cafe)
[10:05 ~ 10:10] taking a break or having a snack (ordering a snack and a drink)
[10:10 ~ 10:12] taking a break or having a snack (and discussing the study group with Isabella Rodriguez)
[10:12 ~ 10:15] taking a break or having a snack (checking her phone and responding to messages)
[10:15 ~ 10:25] taking a break or having a snack (reading through her physics notes)
[10:25 ~ 10:35] taking a break or having a snack (people-watching and observing the cafe atmosphere)
[10:35 ~ 10:45] taking a break or having a snack (enjoying her snack and drink)
[10:45 ~ 10:55] taking a break or having a snack (paying for her snack and drink)
[10:55 ~ 11:00] taking a break or having a snack (walking back from Hobbs Cafe)
可以看到这里细分日程计划被重新安排了,这也改变了故事的走向。
4.3.2 检查path
如果没有任何需要reaction的事件,agent本身的path又不为None,说明agent在赶路,且周围没有任何需要关注的事,那就直接跳过action的决策,否则进入action决策环节。
4.3.3 决定行动(determine_action)
首先判断当前知否有未完成的action,即action结束时间是否已经早于当前时间,如果action仍在进行中就不进行新的action的决策;否则根据当前的plan制定计划:
其实准确地讲,决定行动并没有定制新的行动计划,而是围绕当前时间节点上面的细分日程(decomposed schedule)推断目的地和事件描述,这个过程拆分的相当细,可能是考虑到llm的推理模式很难一下生成所有所需的信息,所以分成了好几步:
首先推断地点,按照从大到小的层级决定sector->arena->object,sector是一个大一点的区域,例如某人的家、咖啡馆、学校等;arena是一个小一点的区域,例如卧室、厨房、图书室等,object就是游戏物品,例如吉他、冰箱、沙发等。一个推断过程如下(有些sector或者arena只有一个子层级,为了节省时间和token就直接跳过推断):
决定sector:
<PROMPT>:
Task -- choose an appropriate area from the area options for a task at hand.
-----
Sam Kim lives in <Sam Kim's house> that has Sam Kim's room, bathroom, kitchen.
Sam Kim is currently in <Sam Kim's house> that has Sam Kim's room, bathroom, kitchen.
Sam Kim enjoy walking around from 8am to 12am.
Area options: <Sam Kim's house, The Rose and Crown Pub, Hobbs Cafe, Oak Hill College, Johnson Park, Harvey Oak Supply Store, The Willows Market and Pharmacy>.
* Stay in the current area if the activity can be done there. Only go out if the activity needs to take place in another place.
* Must be one of the "Area options", verbatim.
Sam Kim is relax in park. For taking a walk, Sam Kim should go to the following area: <Johnson Park>
-----
Jane Anderson lives in <Oak Hill College Student Dormatory> that has Jane Anderson's room.
Jane Anderson is currently in <Oak Hill College> that has classroom, library.
Jane Anderson usually stays in dormitory and read books.
Area options: <Oak Hill College Student Dormatory, The Rose and Crown Pub, Hobbs Cafe, Oak Hill College, Johnson Park, Harvey Oak Supply Store, The Willows Market and Pharmacy>.
* Stay in the current area if the activity can be done there. Only go out if the activity needs to take place in another place.
* Must be one of the "Area options", verbatim.
Jane Anderson is eating dinner. For eating dinner, Jane Anderson should go to the following area: <Hobbs Cafe>
-----
Isabella Rodriguez lives in <Isabella Rodriguez's apartment> that has main room.
Isabella Rodriguez is currently in <Isabella Rodriguez's apartment> that has main room.
Isabella Rodriguez opens Hobbs Cafe at 8am everyday, and stands at the counter until 8pm, at which point she closes the cafe.
Area options: <Hobbs Cafe, Isabella Rodriguez's apartment, The Rose and Crown Pub, Harvey Oak Supply Store, The Willows Market and Pharmacy, Dorm for Oak Hill College, Johnson Park, Ryan Park's apartment>.
* Stay in the current area if the activity can be done there. Only go out if the activity needs to take place in another place.
* Must be one of the "Area options", verbatim.
Isabella Rodriguez is opening Hobbs Cafe and standing at the counter. For greeting and seating customers, Isabella Rodriguez should go to the following area: <
<RESPONSE[0/1]>:
Isabella Rodriguez is opening Hobbs Cafe and standing at the counter. For greeting and seating customers, Isabella Rodriguez should go to the following area: <Hobbs Cafe>
决定object:
<PROMPT>:
Task -- choose most relevant object from the Objects for a task at hand.
[Examples]
-----
Current activity: sleep in bed
Objects: <bed, easel, closet, painting>
The most relevant object from the Objects is: <bed>
-----
Current activity: painting
Objects: <easel, closet, sink, microwave>
The most relevant object from the Objects is: <easel>
-----
Current activity: painting
Objects: <easel, closet, sink, microwave>
The most relevant object from the Objects is: <easel>
-----
Current activity: cooking
Objects: <stove, sink, fridge, counter>
The most relevant object from the Objects is: <stove>
-----
Current activity: watch TV
Objects: <couch, TV, remote, coffee table>
The most relevant object from the Objects is: <TV>
-----
Current activity: study
Objects: <desk, computer, chair, bookshelf>
The most relevant object from the Objects is: <desk>
-----
Current activity: talk on the phone
Objects: <phone, charger, bed, nightstand>
The most relevant object from the Objects is: <phone>
-----
Current activity: greeting and seating customers
Objects: <refrigerator, cafe customer seating, cooking area, kitchen sink, behind the cafe counter, piano>
The most relevant object from the Objects is: <
<RESPONSE[0/1]>:
Current activity: greeting and seating customers
Objects: <refrigerator, cafe customer seating, cooking area, kitchen sink, behind the cafe counter, piano>
The most relevant object from the Objects is: <cafe customer seating>
有了日程计划和游戏物品,就可以对当前的行动(action)进行描述,分别描述角色时间和物品事件:
角色事件:
<PROMPT>:
Task: Turn the input into format (<subject>, <predicate>, <object>).
[Examples]
Input: Sam Johnson is eating breakfast.
Output: (<Dolores Murphy>, <eat>, <breakfast>)
---
Input: Joon Park is brewing coffee.
Output: (<Joon Park>, <brew>, <coffee>)
---
Input: Jane Cook is sleeping.
Output: (<Jane Cook>, <is>, <sleep>)
---
Input: Michael Bernstein is writing email on a computer.
Output: (<Michael Bernstein>, <write>, <email>)
---
Input: Percy Liang is teaching students in a classroom.
Output: (<Percy Liang>, <teach>, <students>)
---
Input: Merrie Morris is running on a treadmill.
Output: (<Merrie Morris>, <run>, <treadmill>)
---
Input: Isabella Rodriguez is greeting and seating customers.
Output: (<
<RESPONSE[0/1]>:
Input: Isabella Rodriguez is greeting and seating customers.
Output: (<Isabella Rodriguez>, <is greeting and seating>, <customers>)
物品事件:
<PROMPT>:
Task: Turn the input into format (<subject>, <predicate>, <object>).
[Examples]
Input: Sam Johnson is eating breakfast.
Output: (<Dolores Murphy>, <eat>, <breakfast>)
---
Input: Joon Park is brewing coffee.
Output: (<Joon Park>, <brew>, <coffee>)
---
Input: Jane Cook is sleeping.
Output: (<Jane Cook>, <is>, <sleep>)
---
Input: Michael Bernstein is writing email on a computer.
Output: (<Michael Bernstein>, <write>, <email>)
---
Input: Percy Liang is teaching students in a classroom.
Output: (<Percy Liang>, <teach>, <students>)
---
Input: Merrie Morris is running on a treadmill.
Output: (<Merrie Morris>, <run>, <treadmill>)
---
Input: cafe customer seating is being used for customers to sit and enjoy their drinks and food.
Output: (<
<RESPONSE[0/1]>:
Input: cafe customer seating is being used for customers to sit and enjoy their drinks and food.
Output: (<cafe customer seating>, <is being used>, <for customers to sit and enjoy their drinks and food>)
当然游戏里面展示的emoji也是用提示词生成的(这是一开始我最想不明白的点,看完代码有种恍然大悟的感觉,原来emoji也在训练语料中啊...):
<PROMPT>:
Convert an action description to an emoji (important: use three or less emojis).
[Examples]
Action description: waking up and starting her morning routine (taking a shower)
Emoji: <200d>♀️
Action description: having breakfast (making coffee)
Emoji: ☕️
Action description: painting (turning on classical music to listen to as she paints)
Emoji:
Action description: exercising (going for a run)
Emoji: <200d>♀️
Action description: having breakfast (putting butter on her toast)
Emoji:
Given the examples above, please convert the following action to an emoji (important: use three or less emojis):
Action description: greeting and seating customers
Emoji:
<RESPONSE[0/1]>:
Action description: greeting and seating customers
Emoji:
4.4 反思(reflect)
每一次感知的时候都会增加poignacy这个数值,查了一下叫痛苦指数,但好像叫记忆重要度更合适,这个指数积累到一定程度后会触发反思。反思过程会追溯最近associate中的记忆和聊天信息,并分别进行总结:
首先从纷繁的记忆库中提取出最关键的几条记忆:
<PROMPT>:
[Information]
0. cafe customer seating is occupied by Klaus Mueller, who is waiting for his food to arrive
1. behind the cafe counter is likely to be in a state of activity, with Isabella Rodriguez interacting with customers, possibly taking orders, making recommendations, and engaging in general customer service. The area might be well-lit, with various tools and ingredients necessary for preparing drinks and food, such as coffee machines, menus, and condiments. The counter itself might be cluttered with items related to the current tasks, such as notepads, pens, and perhaps a tray or two with drinks or snacks. The atmosphere could be bustling and lively, with the sounds of other customers, background music, and the general hustle and bustle of a cafe
2. Klaus Mueller is walking to Hobbs Cafe from the library
3. Maria Lopez is taking a break or having a snack (paying for her snack and drink)
4. cafe customer seating is occupied by Maria Lopez who is taking a break or having a snack (paying for her snack and drink)
5. behind the cafe counter is likely to be in a state of preparation or service, with staff members like Isabella Rodriguez attending to customers, taking orders, and preparing drinks and food. The counter itself might have a cash register open, menus on display, and perhaps some ingredients or tools of the trade visible. The atmosphere would be one of activity and hospitality, with the goal of providing a positive experience for the customers
6. Maria Lopez is taking a break or having a snack (enjoying her snack and drink)
7. refrigerator is being used to store and retrieve food and drinks
8. This is Isabella Rodriguez's plan for Monday February 13: wake up and complete the morning routine at 6:00 AM; eat breakfast at 7:00 AM; prepare for the day at Hobbs Cafe from 7:30 AM to 8:00 AM; open Hobbs Cafe at 8:00 AM and stand at the counter until 8:00 PM; close Hobbs Cafe at 8:00 PM and complete end-of-day tasks until 9:00 PM; return home and relax from 9:00 PM to 11:00 PM; go to bed at 11:00 PM
Given the information above, what are 3 most salient high-level questions?
1.
<RESPONSE[0/1]>:
Based on the information provided, the three most salient high-level questions are:
1. **What is the current status of the cafe counter area?**
- This question addresses the level of activity and the specific tasks being performed behind the cafe counter, which is likely to be a central focus of the scenario.
2. **What is Klaus Mueller doing at Hobbs Cafe?**
- Since Klaus Mueller is mentioned multiple times, it suggests he is an important figure in the context. The question explores his actions and intentions within the cafe.
3. **What is Maria Lopez doing at Hobbs Cafe?**
- Maria Lopez is also mentioned several times, indicating she is another key figure. The question seeks to understand her actions and the context of her visit to the cafe.
然后对提取出来的记忆再次进行检索,找到相关给予碎片进行归纳,绑定高层次的记忆记忆和其根因:
<PROMPT>:
[Statements]
0. behind the cafe counter is likely to be in a state of activity, with Isabella Rodriguez interacting with customers, possibly taking orders, making recommendations, and engaging in general customer service. The area might be well-lit, with various tools and ingredients necessary for preparing drinks and food, such as coffee machines, menus, and condiments. The counter itself might be cluttered with items related to the current tasks, such as notepads, pens, and perhaps a tray or two with drinks or snacks. The atmosphere could be bustling and lively, with the sounds of other customers, background music, and the general hustle and bustle of a cafe
1. behind the cafe counter is likely to be in a state of preparation or service, with staff members like Isabella Rodriguez attending to customers, taking orders, and preparing drinks and food. The counter itself might have a cash register open, menus on display, and perhaps some ingredients or tools of the trade visible. The atmosphere would be one of activity and hospitality, with the goal of providing a positive experience for the customers
2. behind the cafe counter is likely to be a busy and dynamic area with various activities taking place. The counter itself might be cluttered with coffee machines, cups, saucers, sugar bowls, and other serving utensils. There could be a cash register open and ringing up transactions. The air might be filled with the aroma of freshly brewed coffee and baked goods. Isabella Rodriguez would be moving back and forth, multitasking between taking orders, making drinks, and interacting with customers. The overall state would be one of activity and engagement, with a focus on customer service and the preparation of food and drinks
3. behind the cafe counter is likely to be in a state of activity, with various items and tools related to serving customers, handling payments, and providing change. The counter might have a cash register open, with money and receipts, as well as a tray or container for change. There could be a display of menu items or a system for organizing orders. The area might also have a till or safe for storing cash, and there could be a device for processing credit card payments. Isabella Rodriguez would be interacting with these items as she serves customers and manages transactions
4. behind the cafe counter is likely to be in a state of preparation or service, with staff members like Isabella Rodriguez attending to customers, taking orders, and preparing drinks and food. The counter itself might have a cash register open, menus on display, and perhaps some ingredients or tools of the trade like coffee makers, measuring cups, and spoons. The atmosphere would be bustling with activity as the staff works to provide service to the customers
5. cafe customer seating is being used for customers to sit and enjoy their drinks and food
6. This is Isabella Rodriguez's plan for Monday February 13: wake up and complete the morning routine at 6:00 AM; eat breakfast at 7:00 AM; prepare for the day at Hobbs Cafe from 7:30 AM to 8:00 AM; open Hobbs Cafe at 8:00 AM and stand at the counter until 8:00 PM; close Hobbs Cafe at 8:00 PM and complete end-of-day tasks until 9:00 PM; return home and relax from 9:00 PM to 11:00 PM; go to bed at 11:00 PM
7. cafe customer seating is occupied by Isabella Rodriguez, who is serving customers and ensuring the cafe is clean and presentable
8. Isabella Rodriguez is continuing to serve customers at Hobbs Cafe (making coffee and preparing drinks for customers)
9. Maria Lopez and Isabella Rodriguez are excited about the study group tonight and plan to meet at Hobbs Cafe at 6:30 PM to discuss their progress and learn from each other
10. cafe customer seating is available/unoccupied, as Maria Lopez is walking to the cafe, indicating that she has not yet taken a seat. Once she sits down, the seating would be considered occupied
11. cafe customer seating is occupied by Klaus Mueller, who is waiting for his food to arrive
12. Isabella Rodriguez is taking orders and serving customers
13. refrigerator is being used to store and retrieve food and drinks
14. cafe customer seating is occupied by Maria Lopez, who is taking a break or having a snack while people-watching and observing the cafe atmosphere
15. Isabella Rodriguez is greeting and seating customers
16. cafe customer seating is occupied by Maria Lopez who is taking a break or having a snack (paying for her snack and drink)
17. Maria Lopez is taking a break or having a snack (enjoying her snack and drink)
18. refrigerator is being used to store food and drinks, and it is likely running to maintain a cool temperature
19. Klaus Mueller is walking to Hobbs Cafe from the library
20. Maria Lopez is taking a break or having a snack (paying for her snack and drink)
21. Maria Lopez is taking a break or having a snack (reding through her physics notes)
22. kitchen sink is running with water and being used for cleaning purposes
23. Isabella Rodriguez is cleaning the counter and wiping down tables
What 5 high-level insights can you infer from the above statements? (example format: insight (because of 1, 5, 3))
1.
<RESPONSE[0/1]>:
Insight: The cafe counter is a central area of activity and service, with staff members like Isabella Rodriguez engaging in various tasks to serve customers. (Because of 0, 1, 2, 4, 12)
Insight: Isabella Rodriguez has a structured daily routine that involves opening and closing the cafe, as well as serving customers throughout the day. (Because of 6)
Insight: The cafe customer seating is used for customers to enjoy their drinks and food, and it can be occupied by staff members like Isabella Rodriguez when not in use by customers. (Because of 5, 7, 14, 16, 17)
Insight: The refrigerator is an essential tool for storing and retrieving food and drinks, and it is consistently in use to maintain a cool temperature. (Because of 13, 18, 19)
Insight: The kitchen sink is used for cleaning purposes, indicating that maintaining cleanliness is an ongoing task in the cafe. (Because of 22)
如果反思之前发生了对话事件,则对聊天信息进行总结:
<PROMPT>:
[Conversation]
Klaus Mueller: Hey Isabella, it's great to see you. I hope you're having a good day. I was just at the library, working on my research paper about gentrification. It's a fascinating topic, but also quite complex. I was wondering if you had any thoughts or experiences on the matter?
Isabella Rodriguez: Oh, that sounds like an interesting topic for a research paper! I'm not sure I have any specific thoughts on gentrification, but I've definitely seen how neighborhoods can change over time. It's always a bittersweet experience, watching the old give way to the new. Have you found anything particularly surprising in your research so far?
Klaus Mueller: Actually, I've come across some really interesting insights into the unintended consequences of gentrification. For example, while it can bring in much-needed investment and improve living conditions, it can also lead to the displacement of long-term residents and the loss of community character. It's a delicate balance, and I'm trying to understand how these changes affect different groups within a community. What do you think are the most important considerations when discussing gentrification?
Isabella Rodriguez: That's an excellent point. It's crucial to consider the perspectives of all community members, including those who might be adversely affected by gentrification. Have you considered looking into local policies or initiatives that aim to mitigate these negative impacts?
Klaus Mueller: I've definitely considered looking into local policies and initiatives. In fact, I've found some promising examples of community-led efforts that aim to preserve the character of neighborhoods while also encouraging development. It's inspiring to see how communities can come together to shape their own destiny. What do you think are the key factors in ensuring that these initiatives are successful?
Isabella Rodriguez: I think the key factors in ensuring the success of community-led initiatives are strong community engagement, transparent communication, and a commitment to inclusivity. It's important that all voices are heard and that the benefits of development are shared equitably. What are your thoughts on how to foster such an environment?
Klaus Mueller: I think you're right, Isabella. Community engagement and inclusivity are key. It's also important to involve local businesses and organizations in the planning process to ensure that the unique character of the neighborhood is preserved. Have you seen any examples of this in action?
Isabella Rodriguez: It's nice talking with you, looking forward to next time!
Write down if there is anything from the conversation that Klaus Mueller need to remember for her planning, from Klaus Mueller's perspective, in a full sentence.
<RESPONSE[0/1]>:
From Klaus Mueller's perspective, the conversation with Isabella Rodriguez reminded him of the importance of considering the perspectives of all community members, including those who might be adversely affected by gentrification, and the need to foster a community environment that is inclusive, transparent, and committed to preserving the character of the neighborhood while also encouraging development.
总结得到的信息都会被当作thought记录到associate中,对后续决策产生影响。原作中说这里的thought有一个思维层次,也就是想法A可能是由想法B,C,D产生的,这个过程在wounderland中由提示词工程完成,但整个代码读下来好像没找到这个思维层次用在了哪里。。。。可能是读的不够细,有明白的朋友欢迎指导。
4.5 寻路(find_path)
这部分逻辑就比较简单了,就是个标准的动态规划,按照决定的action中主角事件的地址找到对应的tile坐标,然后找一条路绕开所有墙(tile.collision==True)到达目标地点。
5.Quick Start
wounderland提供两种运行机制:页面启动看真实效果和本地模拟用来debug。不管使用哪种模式都需要先安装wounderland:
pip install -r requirements.txt && python setup.py install
5.1 页面启动
首先按照django的标准方式初始化数据库并启动server:
cd playground && python manage.py makemigrations && python mange.py migrate
cd playground && python manager.py runserver
5.1.1 初始化页面
得到的端口号(例如127.0.0.1)加上/village就是游戏初始页面,由于后续想拓展出一些isometric的游戏场景,所以没有绑定index页面为游戏入口,启动后如下:
没有额外操作的情况下每个agent会按照random的方式进行移动,并遵从同样的日程安排。目前默认会启动3个agent模拟,精力有限暂时没有在界面上搞添加agent的功能,可以在:
https://github.com/Archermmt/wounderland/blob/main/playground/game/views.py#L18-L20
游戏开始代码处添加agent名字。
5.1.2 添加llm keys
如果想让agent走脑子,需要配置llm的keys:
首先在游戏左侧User栏中进行用户登录(请自行记录用户名和密码,没有实现任何密码找回功能。。。。当然验证码和邮箱验证就没有,所以user name,user password和user email随便写):
登录之后界面如下(会记录30天的登录状态,如果没有添加keys则不显示LLM Key):
登录之后可以添加LLM Keys
LLM keys的获取方式可以参考:
目前已经支持的llm model和齐对应的keys关系为:
opanai:OPENAI_API_KEY
千帆:QIANFAN_AK && QIANFAN_SK
星火:SPARK_APPID && SPARK_API_SECRET && SPARK_API_KEY
智谱:ZHIPUAI_API_KEY
默认配置为千帆平台的Yi-34B-Chat大模型(不为别的,就为白嫖,花钱的事还是下次一定吧...),所以如果要看到agent思考的效果需要获取QIANFAN_AK和QIANFAN_SK并添加到llm keys中。
配置keys之后agent就可以“走脑子”的进行决策了。游戏设计的时候设计成了准实时模式,也就是agent思考的时候时间会冻结,在地图上行走的时候时间解冻,但由于免费的大模型不能并行调用,所以只能轮流让agent进行思考,那么一开始的几个步骤会看到agent经常“卡”在原地,等其他agent思考完时间解冻之后才能继续行动,这也暴露了llm service的一个问题,响应时间过长,在游戏这种对实时性要求高的地方必须寻求本地大模型推理的解决方案。
5.1.3 agent状态监控
游戏运行起来之后可以打开左侧Agent看板,查看当前agent状态,同时左上角的follow点击之后镜头会一直跟随agent移动,agent看板呈现了agent当前的关键信息:
profile看板展示个人资料,主要包括当前状态(位置,速度,path等)以及个人特性scratch:
memory看板展示agent记忆中比较重要的信息,包括整体associate的统计(事件记忆,思考记忆,聊天记忆各多少条)和聊天的记忆
percept看板展示agent当前感知到的周围信息
plan看板展示agent当前行动action和日程安排schedule
5.1.4 加速游戏时间
在user看板的time一栏可以更改游戏时间,目前支持快进快退10min,加速(最多X 16),这个功能主要也是为了调试使用,由于游戏设计成了准实时模式,不能像原作那样模拟好了再进行呈现,所以增加了time的控制器:
5.2 本地模拟
本地模拟是为了快速debug,配置起来相比页面启动就简单多了,只需要配置一个ckpt.json文件即可:
{
"keys": {
"OPENAI_API_KEY": YOUR_KEY,
"QIANFAN_AK": YOUR_KEY,
"QIANFAN_SK": YOUR_KEY,
"SPARK_APPID": YOUR_KEY,
"SPARK_API_SECRET": YOUR_KEY,
"SPARK_API_KEY": YOUR_KEY,
"ZHIPUAI_API_KEY": YOUR_KEY
},
"keep_storage": false,
"agents": {},
"time": "20230213-09:30",
"step": 0
}
其中的YOUR_KEY部分请自行获取,另外几项含义为:
keep_storage:是否保存agent的记忆,主要就是associate部分,如果设为false则清空associate数据库,默认为true
agents:初始的agents状态,一般不需要配置
time:游戏开始时间
step: 游戏起始步数,用于保存ckpt文件
配置完成后执行:
cd tests && python test_village.py --step 20 --verbose debug --checkpoint ckpt.json
step:模拟步数,每一步之后会保存一个新的ckpt_XX.json,XX为当前步+1
verbose: log等级,debug级别会打印prompt的相关信息,info级别则只打印关键信息
checkpoint:模拟测试开始的ckpt文件,如果想从上一次模拟的断点开始继续模拟,可以换成上一次模拟过程中生成的ckpt文件
每一步模拟获得的ckpt文件会保留全部可以断点加载的信息如:
{
"keys": {
"OPENAI_API_KEY": YOUR_KEY,
"QIANFAN_AK": YOUR_KEY,
"QIANFAN_SK": YOUR_KEY,
"SPARK_APPID": YOUR_KEY,
"SPARK_API_SECRET": YOUR_KEY,
"SPARK_API_KEY": YOUR_KEY,
"ZHIPUAI_API_KEY": YOUR_KEY
},
"keep_storage": true,
"agents": {
"Isabella Rodriguez": {
"status": {
"poignancy": 19
},
"schedule": {
...
},
"associate": {
...
},
"chats": [],
"currently": "Isabella Rodriguez is planning on having a Valentine's Day party at Hobbs Cafe with her customers on February 14th, 2023 at 5pm. She is gathering party material, and is telling everyone to join the party at Hobbs Cafe on *February 14th, 2023, from 5pm to 7pm*.",
"coord": [
67,
14
],
"path": [],
"action": {
"event": {
"subject": "Isabella Rodriguez",
"predicate": "is",
"object": "have lunch",
"describe": "Isabella Rodriguez is have lunch",
"address": [
"the Ville",
"Ryan Park's apartment",
"bathroom",
"bathroom sink"
],
"emoji": "\ud83e\udd81"
},
"obj_event": {
"subject": "bathroom sink",
"predicate": "is",
"object": "idle",
"describe": "bathroom sink is idle",
"address": [
"the Ville",
"Ryan Park's apartment",
"bathroom",
"bathroom sink"
],
"emoji": "\ud83e\udd81"
},
"act_type": "action",
"start": "20230213-12:40:00",
"duration": 10
}
},
"Klaus Mueller":{...},
"Maria Lopez": {...}
},
"time": "20230213-12:50",
"step": 20
}
6.总结与思考
小镇的核心就是提示词工程,所有的思考,行动,决策基本都是用提示词工程完成的。与之配合的是向量数据库,这部分主要体现在associate中,在反思(reflect)和反应(reaction)阶段会检索记忆库得到相关记忆,然后通过prompt将记忆浓缩提炼,方便后续决策过程。整体测试下来,有几点思路可以作为后续优化链路:
6.1 本地大模型部署
agent所有的决策与交谈内容都比较简单,如果不简单,就用few-shot解决。。。。目前看起来对推理准确度没有特别高的要求,但是对于实时性要求极高,目前游戏中agent过一段时间就会停下来等别的agent思考,就是因为一次思考大概调用10-20次llm服务,网络延迟十分可观。为了改善这一体验,可以用本地大模型部署的方式替换现有的qianfan服务,这样agent也可以使用一些非大模型类型的ai model来完成一些简单的task(function calling部分),利于降低资源的消耗。目前在调研ollama和lepton ai准备找一个适合进行本地部署的框架,模型部分正在调研mlc-llm和tensorrt-llm,目前更倾向于mlc,更加轻量简洁,开源能魔改,且适合硬件异构。
6.2 知识库检索
尝试了llamaindex之后,感觉这个库有点盛名之下,其实难副的感觉。。。。。可能是因为我目前的使用场景只是用llamaindex记录agent的思维碎片,基本都是text类型的node,并不是llamaindex的主流使用场景。但是它确实有一些自己的软肋:
1.启动延迟过大:不知道为什llamaindex启动时等待时间极长,不确定是不是我的配置有问题,基本是按照教程来操作:【城南 · LlamaIndex 教程】一文看懂LlamaIndex用法,为LLMs学习私有知识,但import llama_index足足等了好几秒,十分费解在做什么。如果需要加载agent的记忆,启动时间更是恐怖的以分钟计,llamaindex的数据库效率部分实在是不敢恭维。
2.接口变化快:上述教程只是过了几个月,基本llamaindex的所有接口就改的面目全非了,还得一边试一遍看官方教程,这其实也是llm应用框架的通病,几乎都不做后相兼容(langchain,不要看别人就是说你。。。。)。本身接口就奇乱无比,核心部件和contrib部件不分主次、同一作用的接口好几个版本的实现、pipeline规划千变万化、模块调用没有清晰DAG等等就不多说了,关键是过一段还换一次血,搞得维护成本居高不下,希望可以赶紧出现一个工业级的llm应用框架,整顿一下市场。。。。。。
3.llamaindex核心模块耦合度太高:本身应该是llama和index两部分核心组件,二者之间没有必然联系,完全可以作为向量数据库和llm的集合体,每个部分作为一个原子能力互相辅助又能独立使用是最合理的架构模式,但llamaindex偏偏设计成了耦合度极高的模式,甚是费解,可能作者没有考虑会有人像我这样单独使用index功能,然后用prompt实现llama的部分吧。。。。。但如果能做到两者解耦,一定是1+1>2的效果。
这方面正在调研lepton search,毕竟应用基本上搞定了知识库,基本就没太多麻烦的事情了,完全可以自己搞一套langgraph,还更加清晰易懂,并且可以实现radixagent为代表的kv cache的复用机制加速搜索。这部分如果可行会考虑用lepton search进行重构。
6.3 接入metagpt框架
目前wounderland小镇居民目前没有与外界交互的功能,基本都是在自嗨,这也是为什么原作这个很fancy的项目鲜有人继续维护,说白了就是没有商业价值,参考现有ai agent的实现方式,现有构思是做一套任务系统,每个玩家可以发布任务,agent根据自己的人设、能力、记忆领取任务并完成(如果需要协作则通过metagpt类型框架,这里直接重构wounderland的agent即可),玩家在根据每个agent交付的任务结果进行打分,这套系统如果能拓展到联网,基本就是数据收集的利器,当然也可以让agent自己打分,完成自动的数据闭环,当然这种东西也已经有人做了:清华首个AI医院小镇来了,AI医生自进化击败人类专家,数天诊完1万名患者-36氪
这也侧面说明了这个方向的冷饭还是有人炒的。。。。。
6.4游戏场景丰富
虽然目前各个模块已经做了解耦,但是限于人力还是只抄了一遍原作,没有拓展游戏素材,在考虑拓展出一套isometric的游戏场景模式:
目前看phaser是支持这种模式的:
https://examples.ourcade.co/phaser3-typescript/depth-sorting/isometric-map/
不过个人美工经验有限,尚未发现可以测试的完整资源包,有知晓的朋友欢迎介绍!
这个想法不仅仅是为了满足自己童年对GBA游戏的执念,也是可以拓展出更多的环境可变性,因为isometric场景可以看到天空大海,可以看四季更迭,看云卷云舒。。。。。。视野相比与2D的tilemap有极大拓展,这样也方便融入SD之类的算法进行background的绘制,会极大的增加趣味性和游戏丰富度。
6.5 接入上帝视角的角色
“这个世界就是个草台班子”--wounderland搭建了这样一个草台班子,但没有导演。。。。每个角色按照着既定的人生轨迹运行(一开始设定的scratch基本决定了一个agent后续的人生),世界地图也是一成不变的样子,看不到四季交替,看不到人生起伏。如果可以有地图设计者根据当前故事走向重新设计地图,以及一个故事讲述者根据当前情况加入一些环境突发事件来改变接下来的剧情,也会对游戏进展产生极大影响,最终形态可能是根据一部小说或电影,直接完成一部轻量级的游戏。
那几个llm的框架都一言难尽,我去年3月的时候就觉得langchain是一坨,我的项目都是自己写向量检索等模块
嗯,一个围绕prompt的事情搞出来各种各样框架也是醉了,关键还没有哪个架构合理文档清晰的,为了卷而卷了属于是