This will make you a better decision maker | Annie Duke (author of “Thinking in Bets” and “Quit”, former pro poker player)
這將使你成為更好的決策者 | 安妮·杜克（《以賭注思考》和《放棄》的作者，前職業撲克玩家）

Annie Duke (00:00:00): 安妮·杜克 (00:00:00):
It's so incredibly necessary in improving decision quality to take what's implicit and make it explicit. It's not that intuition is crap, your intuition is sometimes right. If you don't make it explicit, then you don't get to find out when it's wrong.
在改善決策質量方面，將隱含的東西變成顯性的東西是非常必要的。直覺並不是一無是處，你的直覺有時是對的。如果你不把它顯性化，那麼你就無法發現它何時是錯的。

Lenny Rachitsky (00:00:12):
When you look at companies that have read your book, what do you find are the brainwashing tactics that really stick?
當你看那些讀過你書的公司時，你發現哪些洗腦策略真的奏效？

Annie Duke (00:00:16):
People generally think the purpose of a meeting is for three things, discover, discuss, decide. The only thing that's ever supposed to happen in a meeting is the discussion part.
人們通常認為會議的目的是三件事，發現、討論、決定。會議中唯一應該發生的事情是討論部分。

Lenny Rachitsky (00:00:25):
Something that comes up in product a lot is this idea of pre-mortems.
在產品開發中經常出現的一個概念是事前檢討。

Annie Duke (00:00:28):
So a pre-mortem, it's great only if you set up kill criteria. Commit to actions that you're going to take if you see those signals.
所以預先檢討只有在你設置了終止標準時才有用。承諾在看到那些信號時要採取的行動。

Lenny Rachitsky (00:00:35):
You have a very interesting framework for how to think about decision quality when the outcome is very long-term.
你有一個非常有趣的框架來思考當結果是非常長期時的決策質量。

Annie Duke (00:00:40): 安妮·杜克 (00:00:40):
There is no such thing as a long feedback loop. And the way you choose to shorten the feedback loop is to say, what are the things that are correlated with the outcome that I eventually desire?
沒有什麼是長反饋循環。而你選擇縮短反饋循環的方式是說，有哪些事情與我最終想要的結果相關聯？

Lenny Rachitsky (00:00:53):
Today, my guest is Annie Duke. Annie is the author of the bestselling book Thinking in Bets, and also her more recent book Quit: The Power of Knowing When to Walk Away. She's also a special partner at First Round Capital, which we spent some time on and is incredibly fascinating. Prior to this part of her career, she was a professional poker player. She's won over $4 million in tournaments, including winning a World Series of Poker Bracelet, and she's the only woman who's won the World Series of Poker Tournament of Champions and the National Poker Heads-Up Championship. Currently, she spends her time helping companies make better decisions. In our conversation, we cover the many lessons that she's learned from her friend Daniel Kahneman, who recently passed away. What simple change she's found has the most impact in a company's ability to make better decisions, how to make better quick decisions when the feedback loop is very long, and also why she doesn't actually believe in long feedback loops, how she changed the way that the partners at First Round Capital make decisions, which is incredibly interesting.
今天，我的嘉賓是安妮·杜克。安妮是暢銷書《以賭注思考》的作者，還有她最近的書《放棄：知道何時離開的力量》。她也是 First Round Capital 的特別合夥人，我們花了一些時間在這上面，這非常有趣。在她職業生涯的這一部分之前，她是一名職業撲克玩家。她在比賽中贏得了超過 400 萬美元的獎金，包括贏得世界撲克系列賽手鐲，她是唯一一位贏得世界撲克系列賽冠軍賽和全國撲克單挑錦標賽的女性。目前，她花時間幫助公司做出更好的決策。在我們的對話中，我們討論了她從最近去世的朋友丹尼爾·卡尼曼那裡學到的許多經驗。她發現的對公司決策能力影響最大的簡單變化是什麼，當反饋循環非常長時如何做出更好的快速決策，以及為什麼她實際上不相信長反饋循環，她如何改變 First Round Capital 合夥人的決策方式，這非常有趣。

(00:01:51):
Plus, why when you're thinking about quitting, that probably means you've already waited too long and you should have quit a while ago. I learned a ton from this conversation and this is definitely going to change the way I think about a lot of things. With that, I bring you Annie Duke after a short word from our sponsors. And if you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. It's the best way to avoid missing future episodes and it helps the podcast tremendously. This episode is brought to you by Vanta. When it comes to ensuring your company has top-notch security practices, things get complicated fast.
此外，當你考慮要放棄時，這可能意味著你已經拖太久了，早該放棄了。我從這次對話中學到了很多，這絕對會改變我對很多事情的看法。接下來，在我們的贊助商短暫插播後，我將為你帶來 Annie Duke 的訪談。如果你喜歡這個播客，別忘了在你最喜愛的播客應用程式或 YouTube 上訂閱和關注。這是避免錯過未來集數的最佳方式，並且對播客有很大的幫助。本集由 Vanta 贊助。當涉及到確保你的公司擁有一流的安全措施時，事情會變得非常複雜。

(00:02:27):
Now you can assess risk, secure the trust of your customers, and automate compliance for SOC 2, ISO 27001, HIPAA and more with a single platform, Vanta. Vanta's market-leading trust management platform helps you continuously monitor compliance alongside reporting and tracking risk. Plus, you can save hours by completing security questionnaires with Vanta AI. Join thousands of global companies that use Vanta to automate evidence collection, unify risk management, and streamline security reviews. Get $1,000 off Vanta when you go to vanta.com/lenny. That's vanta.com/lenny.
現在，您可以使用單一平台 Vanta 來評估風險、確保客戶信任並自動化 SOC 2、ISO 27001、HIPAA 等合規性。Vanta 的市場領先信任管理平台幫助您持續監控合規性，同時報告和追蹤風險。此外，您還可以通過 Vanta AI 完成安全問卷，節省數小時的時間。加入數千家使用 Vanta 來自動化證據收集、統一風險管理和簡化安全審查的全球公司。現在訪問 vanta.com/lenny 可享受 Vanta $1,000 折扣。網址是 vanta.com/lenny。

(00:03:11):
This episode is brought to you by UserTesting. Get fast feedback from real people throughout the development process so that you can build the right thing the first time. Companies are being asked to do more with less. They need to move more quickly while building experiences that meet changing customer expectations, all while minimizing risk and costly rework. With UserTesting, you have a trusted partner who provides user research teams with the skills, tools, and data to be able to articulate the value of user research to the business so that you can make an impact and build the best experiences with confidence. Get started today at usertesting.com/lenny. Annie, thank you so much for being here and welcome to the podcast.
本集節目由 UserTesting 贊助。在開發過程中，從真實用戶那裡獲得快速反饋，讓你第一次就能做對。公司被要求在資源有限的情況下做更多的事情。他們需要更快地行動，同時構建符合不斷變化的客戶期望的體驗，並且要盡量減少風險和昂貴的返工。有了 UserTesting，你就有了一個值得信賴的合作夥伴，為用戶研究團隊提供技能、工具和數據，能夠向企業闡明用戶研究的價值，讓你能夠自信地做出影響並構建最佳體驗。立即訪問 usertesting.com/lenny 開始吧。Annie，非常感謝你來到這裡，歡迎來到我們的播客。

Annie Duke (00:03:58): 安妮·杜克 (00:03:58):
Thank you so much for having me.
非常感謝你邀請我。

Lenny Rachitsky (00:03:59):
It's my pleasure. I was telling a bunch of people that you were coming on this podcast and every single one of them was so excited that we were doing this, so I'm quite excited that this is happening. I want to start with a question about Daniel Kahneman. I know you two were very close. You did a podcast together. I know you grew to be good friends over the past few years. Sadly, he passed away about a week ago at this point.
這是我的榮幸。我告訴了很多人你會來參加這個播客，每個人都非常興奮我們要做這件事，所以我也很興奮這件事正在發生。我想從一個關於丹尼爾·卡尼曼的問題開始。我知道你們兩個非常親近。你們一起做過播客。我知道過去幾年你們成為了好朋友。不幸的是，他大約一週前去世了。

Annie Duke (00:04:21): 安妮·杜克 (00:04:21):
Yeah. 是的。

Lenny Rachitsky (00:04:21):
萊尼·拉奇茨基 (00:04:21):
I'm curious, what are some lasting lessons that you took away from your time with him? Maybe one or two things that stick with you and you think will stick with you?
我很好奇，你從與他相處的時間中學到了哪些持久的教訓？也許有一兩件事讓你印象深刻，並且你認為會一直記得？

Annie Duke (00:04:31): 安妮·杜克（00:04:31）：
It's hard for me to describe properly how humble this man was, and it seems so impossible because his influence was so huge and his intellect was so huge and his insight was so change-making in the areas that he worked. But he was so humble, he really wanted to hear what you thought. He said, "I don't know a lot", he would change his mind. It's funny, I got asked in a class that I was teaching about thinking fast and slow and someone was talking about how there's a lot of studies in there that didn't end up replicating.
我很難恰當地描述這個人有多麼謙虛，這似乎是不可能的，因為他的影響力如此巨大，他的智慧如此巨大，他的洞察力在他工作的領域中是如此具有變革性。但他非常謙虛，他真的想聽聽你的想法。他說：“我知道的不多”，他會改變主意。有趣的是，我在教授一門關於快速與慢速思考的課程時，有人問到裡面有很多研究最終沒有被重複出現。

(00:05:19):
And what I've seen from a lot of other people in the field who really where their work is defined by some line of work that didn't replicate, where they're really arguing against the fact that it doesn't replicate. Just trying to justify or say, "But no", or this and that. This real lack of open-mindedness to the idea that you know what? Maybe it turns out not to be true and that should be okay. But the thing about Danny was when you talked about it, he was the first one to tell you, "I wish priming wasn't in the book. It doesn't replicate." He was just so open-minded to the idea that he knows what he knows now, but he knows that he doesn't know everything.
我看到很多其他領域的人，他們的工作被某些無法重複的研究所定義，他們真的在爭論這些研究無法重複的事實。他們只是試圖辯解或說「但不是這樣」或其他理由，對於這個想法缺乏開放的心態。你知道嗎？也許結果證明這不是真的，這應該是可以接受的。但關於丹尼的事情是，當你和他談論這個問題時，他是第一個告訴你「我希望書中沒有提到啟動效應，因為它無法重複。」他對於自己現在所知道的事情非常開放，但他也知道自己並不是無所不知。

(00:06:06):
Another thing that people don't know about him outside of academics I think are that he was one of the pioneers in something called an adversarial collaboration. So one of his favorite things, and he in some ways invented it, was find someone who really disagrees with you and then write a paper with them and try to figure out with them what's the study that would resolve the issue. So an example of that was Danny did work on happiness as it relates to wealth. Kahneman and a collaborator, Angus Deaton, they published a result that got a lot of press about the relationship between money and happiness, and the idea was money doesn't buy happiness beyond $75,000. So that got published in 2010. I'm sure if you ran that exact same study, it would be more than $75,000 now because it would adjust for inflation. But it was like the idea being once your basic needs are met and you're not that worried about paying rent and so on and so forth, it's not going to have that big of an effect.
還有一件事是學術界以外的人們不知道的，我認為他是對抗性合作的先驅之一。他最喜歡的事情之一，某種程度上也是他發明的，就是找到一個與你意見完全相左的人，然後與他們一起寫一篇論文，試圖找出能解決問題的研究方法。舉個例子，丹尼曾研究過幸福與財富的關係。卡尼曼和他的合作者安格斯·迪頓發表了一個結果，這個結果在媒體上引起了很大的關注，內容是關於金錢與幸福的關係，結論是收入超過 75,000 美元後，金錢不再能買到幸福。這個結果在 2010 年發表。我相信如果現在進行同樣的研究，這個數字會超過 75,000 美元，因為需要考慮通貨膨脹。但這個想法是，一旦你的基本需求得到滿足，不再擔心支付房租等等，金錢對幸福的影響就不會那麼大了。

(00:07:09):
But then a decade later, Matthew Killingsworth said, "That's not true that actually money and happiness are correlated through all the income levels. The more money you have, the more happy you tend to be." Okay, so this is a classic example where people will tear each other down sometimes and that guy's wrong and whatever. And instead, it was Kahneman's like, "All right, let's do it together." And they joined forces along with Barb Mellers to basically try to resolve that issue. He was just always seeking out, it was just like everything was like, why am I wrong? And he was eager and he was excited and he wanted to...
但十年後，Matthew Killingsworth 說：「事實上，金錢和幸福在所有收入水平上都是相關的。你擁有的錢越多，你就越快樂。」好吧，這是一個經典的例子，人們有時會互相攻擊，說那個人錯了等等。然而，Kahneman 的反應是：「好吧，我們一起來做吧。」於是他們與 Barb Mellers 聯手，基本上試圖解決這個問題。他總是在尋求答案，就像一切都是在問，為什麼我錯了？他充滿熱情和興奮，他想要...

(00:07:56):
I remember having a lunch with him where I realized after the lunch that the whole lunch was him asking me questions about my work. Why is he asking me questions about my work? Right? He's Danny Kahneman. But he was just so curious about other people and he just made time for everybody. He just loved the people around him so much. And I think that in the end, it's hard to believe, but I just think that that dwarfs his body of work, which is spectacular. But he as a human being was just so spectacular. And I do think you hear all the time about people who are so incredibly brilliant. It's the brilliant but asshole thing, and it was just so the opposite with him. He was so kind.
我記得有一次和他共進午餐，午餐結束後我才意識到整個午餐時間他都在問我關於我工作的問題。為什麼他要問我關於我工作的問題呢？對吧？他可是丹尼·卡尼曼。但他對別人的事情非常好奇，總是為每個人騰出時間。他非常愛他周圍的人。我認為最終，這很難相信，但我認為這使他的工作成果相形見絀，儘管他的工作成果非常出色。但作為一個人，他實在是太了不起了。我確實認為你經常聽到那些非常聰明的人。這是那種聰明但討厭的事情，而他完全相反。他非常善良。

Lenny Rachitsky (00:08:51):
That's amazing. And it feels like there's probably a strong correlation between people that are curious, always looking for how they're wrong, and finding time to spend it with people to debate and discuss and ask. Feels like there's going to be a strong correlation with that and finding really interesting insights uncovering new things.
這真是太棒了。而且感覺好像在那些好奇心強、總是尋找自己錯誤之處，並花時間與人辯論、討論和提問的人之間，可能存在很強的相關性。感覺這種相關性會與發現真正有趣的見解和揭示新事物密切相關。

Annie Duke (00:09:10): 安妮·杜克 (00:09:10):
Yeah, yeah, yeah. [inaudible 00:09:12].
是的，是的，是的。[聽不清 00:09:12]。

Lenny Rachitsky (00:09:11):
Well, thank you. Thank you for sharing all that. I wish I had known him. Let's talk about decision-making. So there's two areas I want to spend time on. As one would not be surprised, decision-making and quitting. And I know that you've been on a lot of podcasts. I imagine many listeners have read your book or have heard a lot about the stuff that you teach. I'm going to try to come at this from a bunch of different directions and do it a little differently. First of all, I know you're a parent. As last I heard, you have four kids. I just had a kid, he's almost 10 months now.
好的，謝謝。謝謝你分享這些。我真希望我認識他。我們來談談決策吧。有兩個方面我想花點時間討論。這並不令人驚訝，決策和放棄。我知道你上過很多播客。我想很多聽眾都讀過你的書或聽過你教的很多東西。我會嘗試從不同的角度來看這個問題，並以不同的方式來做。首先，我知道你是個家長。據我所知，你有四個孩子。我剛有了一個孩子，他現在快 10 個月大了。

Annie Duke (00:09:43): 安妮·杜克 (00:09:43):
Oh, congratulations. 哦，恭喜。

Lenny Rachitsky (00:09:44):
Thank you. I'm curious, what your frameworks you find most helpful in parenting and in raising your kids that have stuck with you there?
謝謝你。我很好奇，你在養育孩子方面發現哪些框架最有幫助，並且一直堅持使用？

Annie Duke (00:09:53): 安妮·杜克 (00:09:53):
Well, let me just first off say I get a lot of people who are pregnant who come and ask me for my advice about parenting. And I'll tell you the two pieces of advice I give people. The first is this, there's all sorts of parenting books, there's all sorts of styles. A book needs to sell, so they need to sell you something new or different or whatever, okay. But the only thing that matters really is that your kids know you love them, really know deep in their bones, know you love them.
好的，首先我要說的是，很多懷孕的人會來問我關於育兒的建議。我會告訴他們兩條建議。第一是這樣的，有各種各樣的育兒書籍，各種各樣的風格。一本書需要銷售，所以他們需要賣給你一些新的或不同的東西，對吧。但唯一真正重要的是，你的孩子知道你愛他們，真的從心底裡知道你愛他們。

(00:10:32):
And we can argue breastfeeding versus not breastfeeding or co-sleeping versus not co-sleeping or attachment parenting or sleep training, we can talk about all that stuff, right? Do you homeschool? Do you send them to private school? Do you send them [inaudible 00:10:49]? We can have lots of [inaudible 00:10:51], but it all dwarfs in comparison to your children know that you would lay down in the street in the front of a bus if it meant that they would be alive and happy. That's the number one thing I say. Number two is at some point, you will drop your baby on its head. So I bet this has happened to you already.
我們可以爭論母乳餵養與否，或是同床共眠與否，或是依附式育兒或睡眠訓練，我們可以討論所有這些東西，對吧？你是讓孩子在家自學嗎？你送他們去私立學校嗎？你送他們去[聽不清 00:10:49]嗎？我們可以有很多[聽不清 00:10:51]，但這一切都比不上讓你的孩子知道你會為了他們的生命和幸福而躺在公車前的街道上。這是我說的第一件事。第二件事是，在某個時刻，你會不小心讓你的寶寶頭部著地。所以我打賭這已經發生在你身上了。

Lenny Rachitsky (00:11:18):
Well, yeah. Sort of. Okay.
嗯，是的。算是吧。好吧。

Annie Duke (00:11:18): 安妮·杜克（00:11:18）：
Sort of, right. 算是吧，對吧。

Lenny Rachitsky (00:11:18):
Keep coming. 繼續來。

Annie Duke (00:11:20): 安妮·杜克（00:11:20）：
So here's the thing about babies, is babies can't move. And so you get very used to as a mom or a dad or you're sitting on the couch and you're folding laundry and your baby is right here next to you and your baby can't move. And then one day without warning, your baby can roll over. And you think it's fine because you put a pillow there or something, but you turn your back for a second because you're like whatever and then your baby falls on its head. It happened with every single one of my children. And they don't fall that far and whatever, but they're totally fine and you're not a bad parent. And it's just this, there are things you don't know and stuff's going to happen and it's not going to be perfect. And every little thing isn't going to have some lasting impression on your child because at some point, you will drop your baby on its head and you will not mean to, you didn't intend to.
所以關於嬰兒的事情是，嬰兒不能動。所以你會習慣作為媽媽或爸爸，或者你坐在沙發上摺衣服，嬰兒就在你旁邊，嬰兒不能動。然後有一天，沒有任何警告，你的嬰兒可以翻身了。你認為沒關係，因為你放了一個枕頭在那裡或其他東西，但你轉身一秒鐘，因為你覺得無所謂，然後你的嬰兒摔到頭了。這發生在我每一個孩子身上。他們摔得不遠，無所謂，但他們完全沒事，你不是一個壞父母。這只是，有些事情你不知道，事情會發生，不會是完美的。每一件小事都不會對你的孩子產生持久的影響，因為在某個時候，你會不小心讓你的嬰兒摔到頭，你不是故意的，你沒有打算這樣做。

(00:12:21):
But in my case, it was always like, oh my gosh, my baby can roll over. I did not think that was going to happen today. And gosh, by the time I got to the fourth one, it's like I'm chasing after one baby to try to get the diaper on and this and that, so then the other baby hits its head. So I think there's just a lesson in that which is like it's a small thing, don't sweat it and don't get mad at yourself because you made a mistake. It's like the mistake would be, oh, I realize my baby could roll over now. So now you should make sure that you're taking extra precautions about your baby rolling over off the couch. But it's really just a funny way for me to say your baby's going to hit its head or walk into a coffee table or get cut or whatever. It's like they're very resilient and you didn't do anything wrong as a parent, so don't beat yourself up about that. And then I do actually have one other thing I say.
但對我來說，總是像，天啊，我的寶寶會翻身了。我沒想到今天會發生這種事。天啊，等到我有第四個孩子的時候，我就像在追著一個寶寶給他換尿布，這樣那樣的，結果另一個寶寶撞到頭了。所以我覺得這裡有個教訓，就是這是一件小事，不要太在意，也不要因為犯了錯而責怪自己。就像錯誤可能是，哦，我現在知道我的寶寶會翻身了。所以你應該確保你在寶寶翻身時多加小心，不要讓他從沙發上翻下來。但這真的只是我用一種有趣的方式來說，你的寶寶會撞到頭，或撞到咖啡桌，或被割傷等等。他們非常有韌性，你作為父母並沒有做錯什麼，所以不要因此責怪自己。然後我確實還有另一件事要說。

Lenny Rachitsky (00:13:20):
Yeah, go for it. 是的，去做吧。

Annie Duke (00:13:21):
Be having four, you have very little, you have no influence whatsoever basically on what your child's personality is. What you can do as a parent is make sure they say please and thank you. So what I find with parents of one is particularly if that one is very well-behaved, they think they're a great parent and they may be. That would be resulting. They may be, but once you have four, what you realize is like, oh, my parenting really didn't have anything to do with it. But my children do say please and thank you.
有了四個孩子後，你會發現你對孩子性格的影響其實非常有限。作為父母，你能做的就是確保他們會說「請」和「謝謝」。我發現只有一個孩子的父母，特別是如果那個孩子非常乖巧，他們會認為自己是很棒的父母，這可能是真的。但當你有了四個孩子後，你會意識到，其實你的教養並沒有太大影響。但我的孩子們確實會說「請」和「謝謝」。

Lenny Rachitsky (00:14:01):
倫尼·拉奇茨基（00:14:01）：
This is really valuable, good advice. I need to hear this. My wife is going to need to hear this, too. She's always worried about the soft spot on our kid's head and getting hit into some things.
這真是非常有價值的好建議。我需要聽到這些。我妻子也需要聽到這些。她總是擔心我們孩子頭上的軟點會撞到一些東西。

Annie Duke (00:14:09): 安妮·杜克 (00:14:09):
No, it'll be fine. 不，沒事的。

Lenny Rachitsky (00:14:10):
Okay, great. 好的，太好了。

Annie Duke (00:14:11): 安妮·杜克 (00:14:11):
It's actually it's part of why it's there. Really, your baby will be fine. I mean, that's the thing. We used to cart babies around in the wilderness, we were living in case. We're fine, we're built for it. We're built for a much rougher world than the world that babies actually grow up in now. I will say my oldest child when she was in high school, just for whatever reason, she had no interest whatsoever in alcohol or drugs or anything like that. And I remember just feeling so smug about how well I had raised this child and how great she was, and oh, these other parents who clearly weren't doing a good job. I mean, I didn't really but because it's hard not to. And then my next child went through high school and I was like, [inaudible 00:15:02]. That's just a temperament thing.
其實這就是它存在的原因之一。真的，你的寶寶會沒事的。我的意思是，我們以前在荒野中帶著嬰兒四處走動，我們住在洞穴裡。我們沒事，我們是為此而生的。我們是為比現在嬰兒成長的世界更艱難的世界而生的。我想說的是，我的大女兒在高中時，無論出於什麼原因，她對酒精或毒品完全沒有興趣。我記得當時我感到非常自滿，覺得自己把這個孩子教得多麼好，她多麼優秀，哦，這些其他父母顯然沒有做好工作。我的意思是，我並不是真的這麼想，但很難不這麼想。然後我的下一個孩子上了高中，我就像，[聽不清 00:15:02]。這只是性格的問題。

Lenny Rachitsky (00:14:11):
It's a good example of resulting in action.
這是一個導致行動的好例子。

Annie Duke (00:15:08): 安妮·杜克（00:15:08）：
It is a good example of resulting, but all my kids have turned out fine and they're all very different, but yeah. But as far as decision-making is concerned, which is a totally different thing, I'll actually quote Danny Kahneman on this one. Nothing is as important as it seems when you're thinking about it, that's a really important one. One of the things that I used to try to do with my kids all the time was mental time travel, which is actually a very good decision tool. So they would be really upset about something and it could be something that happened at school or it could be like they were grounded for something or whatever.
這是一個很好的結果例子，但我的所有孩子都很好，他們都非常不同，但對。至於決策，這是完全不同的事情，我實際上會引用丹尼·卡尼曼的話。當你在思考時，沒有什麼比它看起來更重要，這是一個非常重要的觀點。我以前經常嘗試和我的孩子們做的其中一件事是心理時間旅行，這實際上是一個非常好的決策工具。所以他們會對某些事情感到非常沮喪，可能是學校發生的事情，也可能是他們因為某些事情被禁足等等。

(00:15:46):
And the thing is when you're in the moment, it just feels like so big. And the thing that I used to say and they say all the time now is this is going to be so great for you when you're 40 at Thanksgiving. You're going to be able to tell these stories to your children and it's going to be the best, like you should be thanking me so that you can talk about your crazy mother at the Thanksgiving table. And it allowed them to get some time and space from it to realize this is going to be funny at some point.
當下的感覺真的很強烈。我以前常說，現在他們也常說，這對你們來說，等到你們四十歲的感恩節時會是很棒的經歷。你們可以把這些故事講給你們的孩子聽，這會是最棒的，你們應該感謝我，這樣你們就可以在感恩節餐桌上談論你們瘋狂的母親。這讓他們有時間和空間去意識到，這在某個時候會變得很有趣。

(00:16:19):
At some point, it's going to be a hilarious story that you tell people. No matter how horrible it seems now, it's going to be a hilarious story at some point when you're older. And it sounds like a fun thing I was doing with them, but it's actually a really good decision-making tool for us in general, right? Look, there are absolutely things that 20 years from now are going to matter, for sure. But most things, no. If you think about most things that ever happen to you, like you're 21 and your girlfriend breaks up with you or whatever and you just think it's the end of the world and you're never going to recover and all these things, but if you could get yourself like, well, how do I really think I'm going to think about this in 10 years? Am I still going to be heartbroken in 10 years?
有一天，這會成為你告訴別人的一個搞笑故事。不管現在看起來多麼糟糕，將來某個時候它會變成一個搞笑的故事。而且這聽起來像是我和他們一起做的有趣事情，但實際上這對我們來說是一個非常好的決策工具，對吧？看，確實有些事情在 20 年後仍然會很重要，這是肯定的。但大多數事情，不會。如果你想想發生在你身上的大多數事情，比如你 21 歲時女朋友和你分手了，或者其他什麼事情，你會覺得這是世界末日，你永遠無法恢復，所有這些事情，但如果你能讓自己想一想，我在 10 年後真的會怎麼看待這件事？我在 10 年後還會心碎嗎？

(00:17:03):
Probably not. And I think that we need to get that perspective of time so that we can get out of the moment because in the moment it just feels so important and the feelings are so big. And it's like the focusing effect on this second and the feelings you're feeling right now are so huge that we forget the scope of time. And I think that that's absolutely one of the best things you can do with your kids, and you can do it in small ways too, right? You can choose to play this video game or you can choose to study for your task. A week from now when you get that test back, how do you think you're going to feel about those two choices? Right?
可能不會。我認為我們需要有時間的觀點，這樣我們才能跳出當下，因為在當下感覺一切都如此重要，情感也如此強烈。就像聚焦效應一樣，這一秒鐘和你現在的感受是如此巨大，以至於我們忘記了時間的範圍。我認為這絕對是你可以對孩子做的最好的事情之一，而且你也可以用小方式來做，對吧？你可以選擇玩這個電子遊戲，或者你可以選擇為你的任務學習。一週後當你拿到那個測驗結果時，你覺得你會如何看待這兩個選擇？對吧？

(00:17:43):
So that was a tool that I used a ton and it's just generally an amazing decision-making tool, just generally for people. It's one that I happen to use with my kids all the time. And then I would say that the other one was using the word nevertheless, which is this is a great leadership skill, the word nevertheless. So let's say my child got caught doing something wrong. I don't know, I found a bag of red solo cups in my backyard that they forgot to throw out after a weekend that I was away. That's hypothetical. So I'm grounding them and it's a lot of argument back, right? So they think this is a debate and they're giving me all of their input and their opinions on why it's unfair and all the other kids do it and the other parents don't get mad and whatever, whatever the argument is. You have to have the balance between them feeling heard, which I think is incredibly important that your children feel heard, and following through on what you know or believe is right.
所以那是一個我經常使用的工具，總體來說是一個很棒的決策工具，對一般人來說也是如此。我經常和我的孩子一起使用這個工具。然後我會說另一個工具是使用「然而」這個詞，這是一個很棒的領導技巧，使用「然而」這個詞。假設我的孩子被抓到做了錯事。我不知道，我在後院發現了一袋紅色的塑膠杯，他們在我週末不在時忘了丟掉。這是個假設。所以我把他們禁足，然後有很多爭論。他們認為這是一場辯論，給我所有的意見和他們認為不公平的理由，說其他孩子也這樣做，其他家長不會生氣，等等，不管他們的理由是什麼。你必須在讓他們感到被聽見和堅持你認為正確的事情之間取得平衡，我認為讓你的孩子感到被聽見是非常重要的。

(00:18:56):
So it's I hear you. Nevertheless, you're grounded for two weeks. I hear what you're saying and I understand. Nevertheless, this is what's going to happen. And obviously, the words that you can use for that might be different. You have so much more authority obviously over your children than you do in other places. But in the workplace, this is very good because employees gripe all the time at decisions that you make in leadership because they think they're right and they want their way. And to have the ability to say, I heard you and your input, trust me, was incorporated into the decision, nevertheless, this is the path we're going to take. Right or wrong, that's what we're going to do. So I think there's a lot of things that basically you can say. These are good things to do with kids, but they're actually just generally really good decision strategies.
所以，我聽到了你的話。然而，你還是被禁足兩個星期。我聽到了你的意見，也理解你的感受。然而，這就是我們要做的決定。顯然，你可以用不同的詞來表達這個意思。對於你的孩子，你顯然有更多的權威。但在職場上，這也是很好的做法，因為員工經常對你在領導中做出的決定抱怨，因為他們認為自己是對的，並且希望事情按照他們的方式進行。能夠說出「我聽到了你的意見，相信我，你的意見已經被納入決策中，然而，這是我們要走的路。」不論對錯，這就是我們要做的。所以，我認為有很多事情基本上可以說，這些是對孩子有益的事情，但實際上它們也是很好的決策策略。

Lenny Rachitsky (00:19:47):
I'm taking notes on all these things. These are going to be extremely useful.
我正在記錄所有這些事情。這些將會非常有用。

Annie Duke (00:19:51):
Nevertheless is a really good one with children.
儘管如此，這對孩子們來說真的很好。

Lenny Rachitsky (00:19:53):
It connects to something Dr. Becky teaches, if you know her, of telling someone, "I believe you." When your kid says something and they're upset about something, start with I believe you.
這與 Dr. Becky 教的東西有關，如果你認識她的話，就是告訴某人「我相信你」。當你的孩子說了什麼並且他們對某事感到不安時，從「我相信你」開始。

Annie Duke (00:20:03): 安妮·杜克 (00:20:03):
Nevertheless. 儘管如此。

Lenny Rachitsky (00:20:03):
萊尼·拉奇茨基 (00:20:03):
And here's nevertheless, exactly. Fair enough. I think we're going to start having to pivot this podcast to a parenting podcast because there's so much stuff here, but I'm going to try to resist. So we've been talking about decision-making and frameworks and things like that. Something I wanted to ask you is how much better can someone get at making better decisions? So for example, say someone listens to all of your podcasts, reads your book, studies it intensely. What's the delta you find in somebody being able to make a better decision? And where [inaudible 00:20:35] comes from is Daniel Kahneman of all people. People ask him, "Do you just live such a optimal life now that you know all of these biases we have and all these mental errors we make?" And he's like, "No, I can't..." I think he said famously, "I don't, actually. I can't use these in practice. This is just stuff that I have learned, but it doesn't actually impact my day to day." So I'm curious just how much better can someone get? What kind of delta have you found in terms of making decisions?
好的，這裡還是要說，沒錯。可以理解。我覺得我們可能需要把這個播客轉型成一個育兒播客，因為這裡有太多相關的內容了，但我會盡量克制。所以我們一直在討論決策和框架之類的東西。我想問你的是，一個人在做出更好的決策方面能提高多少？舉個例子，假設某人聽了你所有的播客，讀了你的書，並且深入研究了這些內容。你發現他們在做出更好決策方面的差距有多大？這個問題的靈感來自於丹尼爾·卡尼曼。人們問他：「既然你知道我們有這麼多偏見和心理錯誤，你的生活是不是變得非常優化了？」他說：「不，我不能……」我記得他著名地說過：「其實我不能。我無法在實踐中運用這些知識。這些只是我學到的東西，但並沒有真正影響我的日常生活。」所以我很好奇，一個人在這方面能提高多少？你發現的決策差距有多大？

Annie Duke (00:20:59): 安妮·杜克 (00:20:59):
Okay, so it depends on whether you actually do the things. I mean, I think that that's what the issue is. I think Kahneman did some work way back that started him on this journey. That was basically work on hiring and it was taking it from completely unstructured Lenny, saying, "I just know a great product manager when I see one", whatever that means, to me going to Lenny and saying, "Okay, I understand that you think that you see a great product manager you know one when you see one", but can you explain to me what that means? In the abstract, what are the things that you're looking for in someone that you want to fill that role? And we can then excavate that, right? And make what is implicit because you're applying some implicit model to how you're thinking about the person that you're interviewing and how they map onto the role that they're going to be, but let's make that explicit.
好的，所以這取決於你是否真的做了這些事情。我的意思是，我認為這就是問題所在。我認為卡尼曼很早以前就做了一些工作，這些工作讓他開始了這段旅程。那基本上是關於招聘的工作，從完全無結構的 Lenny 說「我一看到就知道這是一個很棒的產品經理」，不管那意味著什麼，到我去找 Lenny 說「好吧，我明白你認為你看到了一個很棒的產品經理，你一看到就知道」，但你能不能向我解釋這意味著什麼？抽象地說，你在尋找填補這個角色的人時，會看重哪些特質？然後我們可以挖掘出來，對吧？把你在面試時對這個人的思考方式以及他們如何適應這個角色的隱含模型變成顯性的。

(00:22:04):
So we can make that explicit, we can turn it into a decision rubric, we can create a structured interview process out of that. And then what he found was after you've gone through that process, if you then use your intuition after having done that, not before, that then you actually can really drastically improve your hit rate on hiring. And in that case, it was from about a 50% hit rate to 65. So that's pretty huge, so that's a really big difference. The problem is nobody does it. So I can tell you that most of the conversations that I had with Danny that were about my work in particular were just him saying, "How do you get anybody to do it?" Now that's not to say that I would be better at that than him, it's that he didn't do my job, right? So he was an academic doing research and so on and so forth, I'm living in companies embedding for years.
所以我們可以明確地表達出來，我們可以把它變成一個決策標準，我們可以從中創建一個結構化的面試過程。然後他發現，經過這個過程後，如果你在此之後而不是之前使用直覺，那麼你實際上可以大幅提高招聘的成功率。在那種情況下，成功率從大約 50%提高到 65%。這是非常巨大的差異。問題是沒有人這樣做。所以我可以告訴你，我與丹尼關於我工作的對話大多數都是他在說：「你怎麼讓任何人去做呢？」這並不是說我比他更擅長這個，而是他沒有做我的工作，對吧？他是一個做研究的學者，而我則是在公司裡嵌入多年。

(00:23:02):
I've been with one client for five years, one client for four years, one client for four years also, almost four years, three and a half years. And then I just took on a new client who I've been with for six months, and that's my whole client roster because I stay with them so long. So if you do it, the answer is quite a bit. Like at least in terms of Danny's work in relation to something pretty noisy like hiring, you went from 50% hit rate to 65% hit rate, which is huge. It's enormous. The problem is that the way that you can make decisions in order to be better at them is not supernatural to the way that humans make decisions.
我有一個客戶合作了五年，一個客戶合作了四年，另一個客戶也合作了四年，差不多四年，三年半。然後我剛接了一個新客戶，已經合作了六個月，這就是我所有的客戶名單，因為我和他們合作的時間都很長。所以如果你這麼做，答案是相當多的。就像丹尼的工作與一些相當繁瑣的事情（比如招聘）相比，你的成功率從 50%提高到 65%，這是巨大的。問題在於，為了在決策上做得更好，你所做的決策方式並不是人類做決策的自然方式。

(00:23:47):
I think we think a lot more highly of our intuition than we really ought to. We think very highly of our ability to notice things in the moment and act rationally toward them in a way that really we ought not to. We tend to think that we have insight that other people don't have when actually the other people probably have more insight into our situation than we do. So I think the answer is a lot with a big if, with an if you can get people to do it. And I think that's where the issue comes in. Now, the good news is that you can then reverse that and say if I'm willing to do it, think about what an edge I'm going to have over people who aren't willing to do it, right?
我認為我們對自己的直覺評價過高了。我們過於相信自己能在瞬間察覺事物並理性應對，但事實上我們不應該這麼自信。我們往往認為自己擁有別人沒有的洞察力，但實際上，別人可能對我們的情況有更多的洞察力。所以我認為答案是肯定的，但前提是你能讓人們去做。而這正是問題所在。好消息是，你可以反過來想，如果我願意這麼做，想想我會比那些不願意這麼做的人多出多少優勢，對吧？

(00:24:36):
So we can think there's people who don't know about it, there's people who do know about it but don't do it, and then there's people who do know about it, but do do it. And it's that last group that's so tiny. So the answer is I think a lot, but nobody does it.
所以我們可以認為有些人不知道這件事，有些人知道但不去做，還有些人知道並且去做。而最後這一群人是非常少的。所以我的答案是，我認為很多人知道，但沒有人去做。

Lenny Rachitsky (00:24:54):
倫尼·拉奇茨基 (00:24:54):
So maybe following that thread, when you look at companies that you've worked with or companies that have read your book and really dove in and started to implement some of your piece of advice to make better decisions, if you could look back at the ones that had an impact, what do you find are the mental models or frameworks or tactics that really stick that most often have an impact and the biggest impact?
那麼也許順著這個思路，當你看那些你曾經合作過的公司或是讀過你的書並且真正深入研究並開始實施你的一些建議以做出更好的決策的公司，如果你能回顧那些有影響的公司，你會發現哪些心理模型或框架或策略真正起作用，最常見的有影響力和最大的影響力？

Annie Duke (00:25:18): 安妮·杜克（00:25:18）：
I think the one that's easiest to implement is this. And it's so easy, and I just wish more people would do it. The best way to get somebody's opinion is independently of other people's opinions, independently asynchronously. So the way that I put is I want people to stop talking to each other so much. When we think about what did people generally think the purpose of a meeting is, they think it's for three things. Discovery, which is I want to discover what your opinions are, what your judgments are of something, right? So I want to find out what you think about something. So for example, if we're in product development and we're trying to figure out a timeline, we're trying to develop a product roadmap and we're trying to figure out how long it's going to take to release certain features or something like that, we all come into a room and we start yelling it out together.
我認為這個是最容易實施的。而且它非常簡單，我真希望更多人能這麼做。獲取某人意見的最佳方式是獨立於其他人的意見，獨立且非同步地進行。所以我的意思是，我希望人們少一點互相交談。當我們思考人們通常認為會議的目的是什麼時，他們認為是三件事。探索，也就是我想了解你的意見，你對某件事的判斷，對吧？所以我想知道你對某件事的看法。例如，如果我們在進行產品開發，試圖確定時間表，試圖制定產品路線圖，並試圖確定釋出某些功能需要多長時間，我們都會進入一個房間，然後一起大聲討論。

(00:26:29):
It's so bad for decision-making, I can't even tell you. There's cross influence, the loudest person in the room tends to then have an outsized influence on the decision, the most confident person in the room tends to have an outsized influence on the decision. And that's great if the most confident person is also right, but the problem is that's not always so. And so that's the discovery piece, right? And we tend to do that in a group. And then there's discussion, which is I've now discovered the way that Lenny is modeling this problem or what his judgments are about certain things or his forecasts are, his estimates are. And now, we're going to discuss those ideas and we're going to discuss your ideas in comparison to my ideas, and that's going to happen in the group setting. And then there's also now, we're going to decide in the group setting. So we're going to make some set of decisions about say what the roadmap is going to look like.
這對決策來說非常糟糕，我甚至無法形容。會有交叉影響，房間裡聲音最大的人往往對決策有過大的影響，房間裡最自信的人往往對決策有過大的影響。如果最自信的人也是對的，那很好，但問題是情況並非總是如此。這就是發現的部分，對吧？我們往往會在小組中進行這個過程。然後是討論，我現在發現了 Lenny 是如何建模這個問題的，或者他對某些事情的判斷、預測和估計是什麼。現在，我們將討論這些想法，並將你的想法與我的想法進行比較，這將在小組環境中進行。然後，我們還會在小組環境中做出決定。我們將對例如路線圖會是什麼樣子做出一系列決策。

(00:27:26):
Here's the thing that I think is the easiest to change, is to realize the only thing that's ever supposed to happen in a meeting is the discussion part. So we should absolutely be coming together as human beings to discuss everybody's judgments and opinions and the way they're modeling the problem and their forecasts. In particular, it's really good to come in and discuss the places where people are different. So you've been in many meetings, I'm sure about 80% of the time is double clicking. Oh, I agree. And I want to now we've literally just used different words to say the exact same thing because I'm in agreement.
我認為最容易改變的一點是，會議中唯一應該發生的事情就是討論部分。所以，我們絕對應該作為人類聚在一起，討論每個人的判斷和意見，以及他們如何建模問題和他們的預測。特別是，討論人們意見不同的地方是非常好的。所以你一定參加過很多會議，我敢肯定大約 80%的時間都是在重複點擊。哦，我同意。然後我們現在只是用不同的詞來說完全相同的事情，因為我同意。

(00:28:07):
If we can focus the discussion on places that people disagree, we're much better off. But so now we can take this, discover, discuss, decide. So I'm saying discuss is only supposed to happen in the meeting, that's the only thing that's supposed to happen in the meeting. So what's happening with the other stuff, the discover and the decide? Well, and this is the thing that I think I actually have been able to get people to do is discover what people think before you get in a room independently of each other. So how would you do that? Well, let's imagine that you're going to have a meeting about the product roadmap.
如果我們能將討論集中在人們意見不一致的地方，我們會更有收穫。所以現在我們可以這樣做，發現、討論、決定。所以我說討論只應該在會議中進行，這是會議中唯一應該發生的事情。那麼其他的事情，發現和決定，會怎麼處理呢？這是我認為我實際上已經能夠讓人們做到的事情，就是在進入會議室之前，獨立地發現每個人的想法。那麼你會怎麼做呢？我們來想像一下，你要開一個關於產品路線圖的會議。

(00:28:43):
So you would say to yourself in this meeting, what are the opinions that I need to get from the people in the room? It could be a brainstorm. What are all the different features that we could develop? So it could be a brainstorm. Fine. Write to them independently and say, "Hey, free for off. Just come up with all the different features that you think would be reasonable for us to consider developing and then give me a forced rank from best to worst of your own ideas with some three to five sentence rationale as to why you have these things in this order." And so I could do that, but maybe we already have a list. I can send that out for a forced rank to everybody. Okay, so here's the final list of things we're really considering. Prioritize those for me, just force rank them. And then again, give me a rationale.
所以你在這個會議中會對自己說，我需要從在場的人那裡獲得哪些意見？這可能是一個腦力激盪。有哪些不同的功能是我們可以開發的？所以這可能是一個腦力激盪。好吧。獨立地寫給他們，說：「嘿，自由發揮。只要提出你認為我們應該考慮開發的所有不同功能，然後給我一個從最好到最差的強制排序，並附上三到五句的理由，說明你為什麼會這樣排序。」我可以這樣做，但也許我們已經有一個清單。我可以把這個清單發給大家進行強制排序。好吧，這是我們真正考慮的最終清單。幫我優先排序，強制排序。然後再給我一個理由。

(00:29:32):
Give me a little bit of free writing as to why you think this should be, that you have these things in this order. Maybe we've decided now on what our top five priorities are. Great. I could, before the meeting, send the top five priorities out. "Hey, we've decided on this stuff." For each thing, I'd like to understand what you think a reasonable timeline is, how many sprints it's going to take, so on, so forth, right? So we can ask for that type of information so that we can start making estimates because that's going to affect budget and what we're going back to the board with and so on so forth. But regardless, just figure out what is the thing that we're going to be discussing in this meeting? And I'm going to send it out to everybody independently and I'm going to say, "Don't reply all, just send it back to me independently."
請寫一點自由發揮的內容，說明為什麼你認為應該這樣做，並且按照這個順序來安排這些事情。也許我們現在已經決定了我們的前五個優先事項。太好了。在會議之前，我可以把這五個優先事項發送出去。「嘿，我們已經決定了這些事情。」對於每一件事，我想了解你認為合理的時間表是什麼，需要多少個衝刺（sprints）來完成，等等。這樣我們就可以要求這類信息，開始進行估算，因為這會影響預算以及我們向董事會匯報的內容等等。但無論如何，先弄清楚我們在這次會議上要討論的事情是什麼。我會把這些內容獨立發送給每個人，並且會說：「不要回覆全部，只需獨立回覆給我。」

(00:30:14):
If it's a repeated decision, you can actually create a rubric that lives on Airtable or Coda or Google Sheets or whatever, I don't really care. And people can input their decisions there where they can't see anybody else's decision. So with Google, you can use Google Forms in order to do it, and then it dumps into a spreadsheet that only you can see. That's a great way to do it. So anyway, so you do that and now you can now see everybody's opinions that you then now send out to the group and say, "Everybody, look this over before we come in and discuss." So notice you're still working as a group, but you're working as a group where you're not in the same room together and talking at the same time. And there's a word for that, which is nominal group. So it's a group that at that moment is working independently and asynchronously of each other.
如果這是一個重複的決策，你其實可以在 Airtable、Coda 或 Google Sheets 等平台上建立一個評分表，我不在乎具體是哪個平台。人們可以在那裡輸入他們的決策，而看不到其他人的決策。使用 Google 的話，你可以用 Google 表單來完成，然後它會匯入到只有你能看到的試算表中。這是一個很好的方法。無論如何，你這樣做之後，你就可以看到所有人的意見，然後將這些意見發送給小組，並說：「大家在我們進行討論之前先看看這些意見。」注意，你們仍然是以小組的形式工作，但不是在同一個房間裡同時討論。這種方式有一個專有名詞，叫做名義小組。這是一個在那一刻獨立且非同步工作的團隊。

(00:31:03):
So if you can get people to do that, and I do have companies where I don't consult with them, but I've just come in and talked to them briefly or whatever that do actually implement that piece of it. That is a huge piece of it, that's ginormous. And then you do the same thing for deciding. So the decision should not be made in a room, it's made either. I love the one decision maker model, but not everybody's down with that. But you can have a vote forum where people go vote in private about the way they're leaning.
所以，如果你能讓人們這樣做，我確實有一些公司，我沒有給他們提供諮詢服務，但我只是進來簡短地和他們談過，或者無論如何，他們確實實施了這一部分。這是一個很大的部分，非常重要。然後你在決策時也要做同樣的事情。所以決策不應該在一個房間裡做出，而是應該在其他地方做出。我喜歡一個決策者的模式，但不是每個人都接受這種方式。不過，你可以有一個投票論壇，讓人們私下投票，表達他們的傾向。

(00:31:39):
You can do a variety of things with that, but just don't do it. Don't do it in the meeting. And then just the last thing that I'll add, which is a muscle that you really have to exercise, is I think it's really important to understand that the word alignment in terms of we're all aligned as a group, right? The word alignment is stupid and it shouldn't be used. And I know I'm saying that very harshly, but it's true. It's dumb because it doesn't exist. You have 10 people in a room and they're all really different people with different opinions, and they're never going to come out of the room agreeing with each other. And it's really bad if the expectation is that they're supposed to, and it's really bad for a few reasons. One is it isn't reality, and that I don't like coming out of things without reality actually kicking.
你可以用那個做很多事情，但就是不要做。不要在會議中做。然後我最後要補充的一點是，你真的需要鍛鍊的一個能力，就是我認為理解「一致性」這個詞非常重要，對吧？「一致性」這個詞很愚蠢，不應該使用。我知道我這樣說很嚴厲，但這是事實。這很愚蠢，因為它根本不存在。你有十個人在一個房間裡，他們都是非常不同的人，有不同的意見，他們永遠不會在會議結束後達成一致。如果期望他們會達成一致，那就很糟糕，這有幾個原因。首先，這不符合現實，而我不喜歡在沒有現實基礎的情況下得出結論。

(00:32:38):
So it's not reality. People don't actually agree, they're not actually aligned on the decision. That's just the thing that makes you feel better, right? So I think that that's problem number one. But problem number two is that if the goal is alignment, if the goal is agreement, then the meeting becomes coercive and you never want that. So the way that I'm supposed to talk about my ideas is to convey why I believe what I do, not to convince anybody that I'm right because if I'm working to convince people that I'm right, it becomes coercive and that's horrible. So you have to get comfortable with walking out of the room. This is the nevertheless. Walking out of the room understanding that once you have that discussion, it's not that Lenny, your opinion might change. It could, right? I could say my thing and it could be just so damn brilliant that you change your mind somewhat, right?
所以這不是現實。人們其實並沒有真正同意，他們並沒有真正在決策上達成一致。這只是讓你感覺好一點的東西，對吧？所以我認為這是第一個問題。但第二個問題是，如果目標是達成一致，如果目標是同意，那麼會議就會變成強制性的，而你永遠不希望這樣。所以我應該談論我的想法的方式是傳達我為什麼相信我的觀點，而不是說服任何人我對，因為如果我努力說服人們我對，那就變成了強制性的，這是很糟糕的。所以你必須習慣於走出房間。這就是儘管如此。走出房間後理解，一旦你進行了那次討論，不是說 Lenny，你的意見可能會改變。它可能會，對吧？我可以說我的觀點，它可能是如此的精彩，以至於你會有些改變你的想法，對吧？

(00:33:37):
So you come maybe you're like, "Oh, I'm thinking about this differently and actually", but maybe you don't and you still believe a thing that's very different than me. And leadership has to say, "That's fine. I've heard both of you and I know that this isn't going Annie's way. Nevertheless, trying to think about what all of our goals are, this is what the decision is going to be and it's totally fine that you ended up not agreeing with each other because it's reality. And what it allows me to do is get a better sense of what the space of decisions is." So those things I have been able to get people to do and they're actually quite impactful.
所以你來了，也許你會想，「哦，我在以不同的方式思考這個問題，實際上」，但也許你不會，你仍然相信與我非常不同的事情。領導層必須說，「這沒關係。我聽到了你們兩個的意見，我知道這不是按照安妮的方式進行的。然而，考慮到我們所有的目標，這就是最終的決定，並且你們最終沒有達成一致是完全可以的，因為這是現實。這讓我能夠更好地了解決策的範圍。」所以這些事情我已經能夠讓人們去做，實際上它們相當有影響力。

Lenny Rachitsky (00:34:17):
Imagine a place where you can find all your potential customers and get your message in front of them in a cost-efficient way. If you're a B2B business, that place exists, and it's called LinkedIn. LinkedIn Ads allows you to build the right relationships, drive results, and reach your customers in a respectful environment. Two of my portfolio companies, Webflow and Census, are LinkedIn success stories. Census had a 10X increase in pipeline with the LinkedIn startup team. For Webflow, after ramping up on LinkedIn in Q4, they had the highest marketing source revenue quarter to date. With LinkedIn Ads, you'll have direct access to and can build relationships with decision-makers, including 950 million members, 180 million senior execs and over 10 million C-level executives. You'll be able to drive results with targeting and measurement tools built specifically for B2B. In tech, LinkedIn generated two to five X higher return on ad spend than any other social media platforms.
想像一個地方，你可以找到所有潛在客戶，並以成本效益高的方式將你的訊息傳達給他們。如果你是 B2B 企業，那個地方就存在，它叫做 LinkedIn。LinkedIn 廣告讓你能建立正確的關係、推動成果，並在一個尊重的環境中接觸你的客戶。我的兩家投資組合公司，Webflow 和 Census，都是 LinkedIn 的成功案例。Census 在 LinkedIn 初創團隊的幫助下，銷售管道增加了 10 倍。對於 Webflow，在第四季度加大 LinkedIn 的投入後，他們創下了有史以來最高的營銷來源收入季度。通過 LinkedIn 廣告，你將能直接接觸並建立與決策者的關係，包括 9.5 億會員、1.8 億高級主管和超過 1000 萬 C 級高管。你將能使用專為 B2B 設計的定位和測量工具來推動成果。在科技領域，LinkedIn 的廣告投資回報率比其他任何社交媒體平台高出兩到五倍。

(00:35:14):
Audiences on LinkedIn have two times the buying power of the average web audience, and you'll work with a partner who respects the B2B world you operate in. Make B2B marketing everything it can be and get $100 credit on your next campaign. Just go to linkedin.com/podlenny to claim your credit. That's linkedin.com/podlenny. Terms and conditions apply. There's an implication here you touched on a bit that there's a DRI essentially, there's one decision maker. Sometimes people start to feel like, "Oh, my voice isn't heard. I don't have a lot of say, I can't be part of this decision." And you talked a bunch about just how to make people feel included, you get feedback along the way. But any advice there if you try to move to this model of making people feel like, "Okay, I actually have impact on where this goes"?
LinkedIn 的受眾擁有兩倍於一般網路受眾的購買力，並且您將與一位尊重您所處 B2B 世界的合作夥伴一起工作。讓 B2B 行銷發揮其最大潛力，並在您的下一個活動中獲得 $100 的信用額度。只需前往 linkedin.com/podlenny 來領取您的信用額度。網址是 linkedin.com/podlenny。適用條款和條件。這裡有一個您稍微提到的暗示，即有一個主要負責人（DRI），即一個決策者。有時人們會開始覺得，「哦，我的聲音沒有被聽到。我沒有太多發言權，我不能參與這個決策。」您談了很多關於如何讓人們感覺被包含在內，並在過程中獲得反饋。但如果您嘗試轉向這種模式，讓人們感覺「好吧，我確實對這個方向有影響力」，有什麼建議嗎？

Annie Duke (00:35:58): 安妮·杜克 (00:35:58):
Yeah. So here's the really wonderful thing about moving from a coercive model to really, I guess you could say a model of curiosity, right? You want to be curious, not coercive, which means that the way that people in the meeting are talking about what their opinions are is in the mode of conveying information, not trying to convince anybody. So once we move away from that coercive model, and when I say coercive, I'm not saying anybody is purposely trying to set up a culture of coercion. Sometimes that's true, but for the most part, everybody's trying to do a good job and nobody's trying to set up a culture of coercion. But as soon as you say, "Are we all in agreement? Are we all in alignment?" And as soon as you're allowing people, and I'm sure you've been in these meetings, right? If you allow people to interrupt, if you allow people to say this, "I think you're wrong", "I disagree", "Here's why", those are all very coercive things to have happen.
是的。所以，從一個強制模型轉變到一個好奇模型，這真的是一件非常美妙的事情，對吧？你想要的是好奇，而不是強制，這意味著會議中的人們在談論他們的意見時，是在傳達信息，而不是試圖說服任何人。所以，一旦我們遠離那種強制模型，當我說強制時，我並不是說有人故意設立一種強制文化。有時確實如此，但大多數情況下，每個人都在努力做好工作，沒有人試圖設立一種強制文化。但只要你說，“我們都同意嗎？我們都一致嗎？”而且只要你允許人們，我相信你也參加過這樣的會議，對吧？如果你允許人們打斷，如果你允許人們說，“我認為你錯了”，“我不同意”，“這是為什麼”，這些都是非常強制的事情。

(00:37:12):
Interrupting someone is silencing them. Saying, "You're wrong", well now, you become tribal and people aren't going to be open-minded and they're going to stake their ground and it's all really bad. So let's move away from that. Number one, that's already going to help. But when we think about the way that meetings normally happen, again, there's all this crosstalk and some people aren't speaking and some people are and so on, so forth. And then of course, not everybody feels equally heard. But when you work as a nominal group, let's talk about something as simple as we're going to make a forecast of how long it's going to take to launch this product feature. Okay. So I'm going to send out to everybody, what's your point estimate? What's your lower bound? What's your upper bound in terms of timeline?
打斷別人就是讓他們閉嘴。說「你錯了」，這樣你就變得部落化，別人就不會開放心態，他們會堅持自己的立場，這一切都很糟糕。所以我們要遠離這種做法。首先，這已經會有所幫助。但當我們考慮會議通常是怎麼進行的時候，又會有很多交叉對話，有些人不說話，有些人說話，等等。因此，並不是每個人都感覺到同樣被聽見。但是當你以名義小組的方式工作時，讓我們談談一些簡單的事情，比如我們要預測推出這個產品功能需要多長時間。好吧。所以我要發給每個人，問他們的點估計是什麼？他們的下限是什麼？他們的上限是什麼？

(00:38:02):
And you could do it in some way, like how many sprints do you think is going to take whatever language you want to use? Now, everybody independently now gives their forecast with a rationale for why they believe that. And then you come into the room and you run a discussion where everybody's getting to say what their estimate was and why they believe that, and people are getting to ask questions. "I have a question" always is a clarification, it's just I don't understand. And as the leadership, the way that you would do it is, let me give you an example from a real one because that'll be easy. I did one of these discussions for a question about remote work, like what did the company want in terms of remote versus hybrid and that kind of thing. And so there was a lot of disagreement about whether whatever the policy was, it should be consistent across functions. Lots of disagreement.
你可以用某種方式來做這件事，比如你認為需要多少次衝刺來完成你想用的語言？現在，每個人都獨立地給出他們的預測，並解釋他們為什麼這麼認為。然後你進入會議室，進行一場討論，讓每個人都能說出他們的估計和理由，並讓大家提問。"我有個問題"通常是澄清問題，只是我不明白。而作為領導，你會這樣做，讓我舉個真實的例子，這樣會比較容易。我曾經為一個關於遠程工作的問題進行過這樣的討論，比如公司對遠程和混合工作的需求是什麼。對於政策是否應該在各個職能部門之間保持一致，存在很多分歧。

(00:39:01):
So let's take somebody who was on the, no, it shouldn't be consistent across functions. So they now say why they believe it shouldn't be consistent across functions and they say things like, "Well, different functions have different requirements, right?" So there's some functions that have to be in the office, like if you're IT as an example, but there's other functions which are more collaborative and creative and whatnot where it makes sense for people to be in the office, whereas engineers doesn't really matter, right? So there's different needs of different functions in terms of how much they need to be in the same space. So I'm not agreeing or disagreeing with this, I'm just saying what somebody said. So now as a facilitator, I never say, "Oh, I agree." What I say is I just want to make sure I understood what you said and I reflect it back.
所以，讓我們來看看某個人的觀點，他認為不應該在各個職能部門之間保持一致。他們解釋為什麼不應該一致，並說道：「不同的職能部門有不同的需求，對吧？」有些職能部門必須在辦公室工作，比如 IT 部門，但有些職能部門更具協作性和創造性，這些部門的人員在辦公室工作是有意義的，而工程師則無所謂。所以，不同職能部門對於在同一空間工作的需求是不同的。我並不是在同意或反對這個觀點，我只是陳述某人的說法。作為一個主持人，我從不說「我同意」，我會說「我只是想確認我理解了你的意思」，然後再反映回去。

(00:39:57):
So I say, "So what I heard you say is that not all functions are created equal in terms of how well they work remotely or how well they work in person or what the flexibility might be. So what you're saying is there're functions that have to be in person. Period. And then other functions where that collaborative element being in the same office would be more important versus some functions where being remote is totally fine. Is that what you meant?" And then they have the ability to say, "Yes, that is what I meant" or, "Actually, I meant something slightly different. Let me say what that is", and then I reflect that back. Okay, so literally you're just going around, you're calling on people to do that, and then you're reflecting back what they say without offering your own opinion. I don't know, Lenny. Tell me how someone doesn't feel heard in that situation.
所以我說：「所以我聽到你說的是，不是所有的職能在遠程工作或面對面工作時的效果都是一樣的，或者它們的靈活性也不一樣。你是說有些職能必須要面對面工作。就這樣。然後有些職能在同一個辦公室裡的協作元素更重要，而有些職能遠程工作完全沒問題。這是你的意思嗎？」然後他們可以說：「是的，這是我的意思」或者「其實，我的意思有點不同。讓我來解釋一下」，然後我再反映回去。好吧，所以你基本上就是這樣，叫人們這樣做，然後反映他們說的話而不提供自己的意見。我不知道，Lenny。告訴我在這種情況下怎麼會有人覺得自己沒有被聽到。

Lenny Rachitsky (00:40:48):
倫尼·拉奇茨基 (00:40:48):
Yeah. Absolutely. I would feel so good if somebody just clarified and made it clear they know exactly what I'm saying, even if the decision doesn't end up the way I want.
是的，絕對的。如果有人能夠澄清並讓我知道他們完全理解我在說什麼，即使最終的決定不是我想要的，我也會感到非常好。

Annie Duke (00:40:57): 安妮·杜克 (00:40:57):
Right, so that's the thing. And then what actually ends up happening is that the people in the room feel more, the psychological term would be endowed, more endowed to the decision. In other words, they feel like they have ownership over the decision. And whatever the decision is, they generally will see their selves reflected in it because they were heard. And they also will generally understand that nobody, no one really ends up with exactly every single thing that they wanted to see in the decision because they also get to see the true spread of opinions on the team. And what you see in that situation is that there's lots and lots and lots and lots of disagreement, which you don't see if you talk in a group. It narrows the space, right?
對，這就是問題所在。然後實際上發生的事情是，房間裡的人會感覺更有心理上的賦權，換句話說，他們覺得自己對這個決定有所有權。無論決定是什麼，他們通常會在其中看到自己的影子，因為他們被聽到了。他們也通常會理解，沒有人能完全得到他們想要的每一件事，因為他們也看到了團隊中意見的真實分佈。在這種情況下，你會看到有很多很多的分歧，而這在小組討論中是看不到的。這縮小了空間，對吧？

(00:41:48):
So for example, particularly if you're senior, if you say, "I think this is going to take three months to launch" and I was thinking four weeks, you're never going to hear four weeks from me, ever. But if we get those opinions independently, you will actually hear that I think it's going to take four weeks. You may tend to be more right there being more senior, but I may have something interesting to say. You should hear what I have to say. And this also allows me to learn from you, too. But because I haven't heard your opinion first, then I'm not going to conform my opinion to yours. So it actually spreads the surface area of disagreement that you see on the team, which then makes people feel much better about the decision not being exactly what they want because they recognize like, "Oh, this is actually a hard problem. People really disagree on this stuff."
舉個例子，特別是如果你是資深員工，如果你說「我認為這需要三個月才能推出」，而我原本認為是四週，你永遠不會從我這裡聽到四週這個時間。但如果我們獨立地表達意見，你會實際上聽到我認為需要四週。你可能因為更資深而更有可能是對的，但我可能也有一些有趣的見解。你應該聽聽我的意見。這也讓我能從你那裡學到東西。但因為我沒有先聽到你的意見，所以我不會把我的意見調整成和你一樣。這實際上擴大了團隊中你所看到的分歧範圍，這讓人們對決策不是完全符合他們的期望感到更好，因為他們認識到「哦，這其實是一個難題。大家對這些事情真的有不同意見。」

Lenny Rachitsky (00:42:41):
Amazing. I think this is going to be really helpful to a lot of people. Just to close the thread on this little summary maybe is core advice here is brainstorm separately. So I guess first, there's discover, discuss, decide. To discover ideas completely independently, basically brainstorms. You're a big advocate of brainstorming independently, sitting on your own, thinking through ideas. And then bringing people together to discuss all the things they've come up with and especially where they disagree. And then ideally having one person that makes a decision once she or he has taken all the input.
太棒了。我認為這對很多人來說會非常有幫助。為了結束這個小總結，也許這裡的核心建議是分開腦力激盪。所以我猜首先是發現、討論、決定。完全獨立地發現想法，基本上就是腦力激盪。你非常提倡獨立腦力激盪，自己坐下來思考想法。然後把人們聚集在一起討論他們提出的所有想法，特別是他們不同意的地方。然後理想情況下，有一個人在收集了所有意見後做出決定。

Annie Duke (00:43:12): 安妮·杜克（00:43:12）：
Yeah, you can have one person. I mean, I work with people where it's like a partnership and there's six people who are going to vote, but they still do it independently. They go to a forum. So I don't know what your vote is. And all six people don't have to agree. I mean, I just think it's really important that once you get above an end of one, you shouldn't necessarily expect the people to agree, which I think is just really important. And this is good for more than just brainstorming. It's for forecasting, any kind of project planning, budgeting. Yeah, I mean, we do this, for example, at First Round, we have a structured forum for evaluating a company in terms of whether you should invest in it because there are facts and then there are the way that you model those facts in. So it's a lot, what are the investor's opinion of the founder and the product and that kind of thing.
是的，你可以有一個人。我是說，我和一些人合作，就像一個合作夥伴關係，有六個人要投票，但他們仍然是獨立進行的。他們去一個論壇。所以我不知道你的投票是什麼。而且六個人不必都同意。我是說，我只是覺得一旦你超過一個人的情況，你不應該期望所有人都同意，這點非常重要。而且這不僅僅對頭腦風暴有用，對於預測、任何類型的項目規劃、預算編制也很有用。是的，我是說，我們在 First Round 就這樣做，我們有一個結構化的論壇來評估一家公司是否值得投資，因為有事實，然後還有你如何建模這些事實。所以這涉及很多，投資者對創始人和產品的看法等等。

(00:44:06):
And we just have structure around how we're eliciting those opinions, which is really helpful. The other thing that I'll just add to what you said is that you can actually be in the same room together and still discover information independently. So this will happen on the fly all the time where people are talking and they start talking. Something comes up, like someone suggests some new feature or something like that, and then people start saying, "Yeah, but that's going to take..." And you go, "Stop. Okay, everybody take out a piece of paper." So you can still get that same independence on the fly as well as doing it in advance. But yes, that was a very nice summary.
我們只是對如何引導這些意見有了一個結構，這真的很有幫助。我還想補充一點，就是即使你們在同一個房間裡，也可以獨立發現信息。這種情況經常會即興發生，人們在交談時，某人提出了一些新功能之類的建議，然後大家開始說：「是的，但這會花費……」然後你說：「停。好，大家拿出一張紙。」所以你仍然可以即興獲得同樣的獨立性，也可以提前進行。但總的來說，這是一個非常好的總結。

Lenny Rachitsky (00:44:49):
Lenny Rachitsky（00:44:49）：
If this is just the one thing people take away from this conversation already, I think that could have a big impact. Speaking of First Round, one, we actually just had Todd Jackson on the podcast talking about-
如果這是人們從這次對話中唯一得到的收穫，我認為這已經會產生很大的影響。說到 First Round，我們其實剛剛在播客上邀請了 Todd Jackson 談論-

Annie Duke (00:44:59): 安妮·杜克 (00:44:59):
Love Todd. 愛托德。

Lenny Rachitsky (00:44:59):
萊尼·拉奇茨基（00:44:59）：
... love Todd, talking about product market fit. That episode will have come out before this episode. Also, so I asked Brett Berson, your colleague at First Round, what to ask you.
... 愛托德，談論產品市場契合度。那集節目會在這集節目之前播出。另外，我問了你的同事布雷特·伯森在 First Round 應該問你什麼問題。

Annie Duke (00:45:00): 安妮·杜克 (00:45:00):
Oh, okay. 哦，好吧。

Lenny Rachitsky (00:45:08):
萊尼·拉奇茨基 (00:45:08):
Yeah, yeah. By the way, your title of First Round is amazing, special partner. I've never seen that before.
是的，是的。順便說一下，你的第一輪頭銜很棒，特別的夥伴。我以前從未見過這樣的。

Annie Duke (00:45:15): 安妮·杜克 (00:45:15):
I am. It's a title just for me.
是的。這是一個只屬於我的頭銜。

Lenny Rachitsky (00:45:17):
倫尼·拉奇茨基 (00:45:17):
Oh, my God. 天啊。

Annie Duke (00:45:17): 安妮·杜克（00:45:17）：
I adore Brett Berson. 我喜愛布雷特·伯森。

Lenny Rachitsky (00:45:19):
Lenny Rachitsky（00:45:19）：
Me, too. 我也是。

Annie Duke (00:45:19): 安妮·杜克（00:45:19）：
So I'm interested, I'm interested in what he said you should ask me.
所以我有興趣，我有興趣知道他說你應該問我的問題。

Lenny Rachitsky (00:45:23):
Nothing too spicy, it's along the lines of what you talked about. So you said that you have a very interesting framework for how to think about decision quality in the short term when the outcome is very long term. For example, investing. Also, many decisions we make in business, things you need to decide now that you only find out years from now. Can you talk about your advice there and your insights here?
沒有什麼太刺激的，這與你談到的內容有關。所以你說你有一個非常有趣的框架來思考短期內的決策質量，而結果是非常長期的。例如，投資。還有，我們在商業中做出的許多決策，現在需要決定的事情，幾年後才會知道結果。你能談談你的建議和見解嗎？

Annie Duke (00:45:43):
Oh my gosh, I'm so happy that that's the question that he asked.
天啊，我真高興他問了這個問題。

Lenny Rachitsky (00:45:45):
Yep, I see. 是的，我明白了。

Annie Duke (00:45:47): 安妮·杜克（00:45:47）：
Okay, so can I give a tiny bit of background to this?
好的，那我可以稍微介紹一下背景嗎？

Lenny Rachitsky (00:45:49):
Absolutely. 當然。

Annie Duke (00:45:50): 安妮·杜克（00:45:50）：
So prior to talking to First Round, I have another client too who, they're amazing, Renegade Partners at Roseanne Wincek and Renata Quintini. They're incredible. Before finally hooking up with them, and they work at different stages, First Round is C, obviously. Renegade is more like A B, tiny bit of dabble in C. But before running into them, I talked to quite a few venture firms who are interested in talking to me post-Thinking in Bets having come out, so this would be 2018. And there was a theme, there was a theme across them all. The first one was, well, the kind of decision-making you're talking about we don't need to do because we just know a good founder when we see one. So that is a sentence that came out of many people's mouths. And as I just said to you, okay, I have no doubt, but don't you want [inaudible 00:46:52] make that explicit?
在與 First Round 交談之前，我還有另一個客戶，他們很棒，Renegade Partners 的 Roseanne Wincek 和 Renata Quintini。他們非常出色。在最終與他們合作之前，他們在不同階段工作，First Round 顯然是 C 輪。Renegade 更像是 A B 輪，稍微涉足 C 輪。但在遇到他們之前，我與一些風險投資公司交談過，他們在《Thinking in Bets》出版後對與我交談感興趣，所以這應該是 2018 年。這些公司之間有一個共同的主題。第一個是，嗯，我們不需要做你所說的那種決策，因為我們一看到好的創始人就知道。所以這句話從很多人嘴裡說出來。正如我剛才對你說的，好吧，我毫不懷疑，但你不想讓[聽不清 00:46:52]明確嗎？

(00:46:53):
There's all sorts of great things that come from making it explicit, not just in terms of the increase in decision quality in the moment, but it actually allows you to close feedback loops much better. So that was one thing that I found quite surprising. But the one that I really found very interesting was being told, "Well, what you're talking about doesn't apply because our feedback loops are a decade." And in poker, you got an answer right away. You won or lost the hand right away. So the way that you're thinking about decision-making doesn't really apply, until I met First Round and then Renegade where they actually heard what I had to say because I gave the same answer to everybody. So we'll just put aside that wouldn't you want to make that explicit. The first thing that I would say is, oh, poker is much noisier than you think because when I win a hand, I have no idea why.
將事情明確化帶來了各種好處，不僅能在當下提升決策質量，還能更好地關閉反饋迴路。這是我發現的一個相當令人驚訝的點。但我真正覺得有趣的是，有人告訴我：「你所說的並不適用，因為我們的反饋迴路需要十年。」而在撲克中，你會立即得到答案，你會立即知道這手牌是贏還是輸。所以你對決策的思考方式並不適用，直到我遇到 First Round 和 Renegade，他們實際上聽取了我的意見，因為我給每個人的答案都是一樣的。所以我們先不談你是否想要將其明確化。我首先會說，哦，撲克比你想像的要更嘈雜，因為當我贏了一手牌時，我根本不知道為什麼。

(00:47:50):
So I do actually have to wait a long time because I have to play many, many, many hands before I actually know, do I actually have an edge? Because I actually don't know very much. On one hand, for one thing, I almost never see my opponent's card, so I'm left in a dust of uncertainty. But separately from that, the main thing that I said was, how could you possibly think that the feedback loop is 10 years? And this is what I think really caught First Round's eye because when I was talking to Josh Kopelman about it, he said, "Well, what do you mean? We don't get an exit for 10 years." And I said, "Oh, I'm sorry, do you invest? And then you go to sleep like Rip Van Winkle? And then you wake up 10 years later and you go, 'Hey, how'd that go?' Or are there all sorts of things that happen in between?" The simplest thing, the simplest thing is does it fund at Series A?
所以我確實需要等待很長的時間，因為我必須玩很多很多很多手牌，才能真正知道我是否有優勢。因為我其實知道的不多。一方面，我幾乎從未見過對手的牌，所以我處於不確定的狀態。但除此之外，我主要說的是，你怎麼可能認為反饋循環是十年？這正是我認為真正引起 First Round 注意的地方，因為當我和 Josh Kopelman 談論這個問題時，他說：「你是什麼意思？我們十年內都沒有退出。」我說：「哦，對不起，你是投資後就像 Rip Van Winkle 一樣去睡覺，然後十年後醒來說，‘嘿，結果怎麼樣？’還是中間發生了各種各樣的事情？」最簡單的事情，最簡單的事情就是它是否在 A 輪融資。

(00:48:46):
And the little pushback that I would get there is, but we're not investing for Series A. And I say, "Well, I know that", but have you ever had a company that exited for more than a billion dollars that did not fund at Series A? And the answer is no. And I'm like, okay. So it sounds like that's necessary. Might not be sufficient, but it's necessary. And it's certainly a signal that is actually more highly correlated with exiting out well than the investment at C. And then, oh, right, you have series B. And that's separate from all the other things that you can look at, like what you talked to Todd Jackson about. Is it achieving product market fit? We know that eventually for it to be successful, it's going to have to achieve product market fit, right? So you can look at what's happening with that, just general things about traction, what's happening with net new ARR, ability to retain top talent churn.
我會遇到的一點反對意見是，我們不是在為 A 輪投資。而我會說，「我知道」，但你有沒有遇過一家公司在沒有 A 輪融資的情況下退出並超過十億美元的？答案是否定的。我就會說，好吧。聽起來這是必要的。可能不是充分條件，但它是必要的。而且這確實是一個比 C 輪投資更高度相關的信號。然後，哦，對，你還有 B 輪。這與你可以考慮的其他所有事情是分開的，比如你和 Todd Jackson 談到的。它是否達到了產品市場契合度？我們知道最終要成功，它必須達到產品市場契合度，對吧？所以你可以看看那方面的情況，還有一些關於牽引力的普遍情況，淨新增年度經常性收入的情況，保留頂尖人才的能力流失情況。

(00:49:55):
I mean, there's so many different things that you can look at, all of which are things that you know must happen in order for the big thing to happen. Okay, so what that means is that this is the big, I'm going to make a bold statement here. There is no such thing as a long feedback loop. You can make a decision about how long the feedback loop is. That is your choice to live in a long feedback loop, and you can choose to shorten the feedback loop. And the way you choose to shorten the feedback loop is to say, what are the things that are necessary but not sufficient? That's one thing, for getting a good exit, or what are the things that are correlated with the outcome that I eventually desire? And what that means is that when you're at the decision point, right? Like in First Round's case, I'm going to invest in a company.
我的意思是，有很多不同的事情你可以看，所有這些事情都是你知道必須發生的，才能讓大事發生。好吧，這意味著這是個大事，我要在這裡做一個大膽的聲明。沒有什麼叫做長反饋循環。你可以決定反饋循環的長短。選擇生活在長反饋循環中是你的選擇，你也可以選擇縮短反饋循環。而你選擇縮短反饋循環的方法是說，哪些事情是必要但不充分的？這是一件事，為了獲得好的退出，或者哪些事情與我最終想要的結果相關聯？這意味著當你在決策點時，對吧？就像在 First Round 的情況下，我要投資一家公司。

(00:50:51):
What you have to understand is that you are making a prediction about how the world is going to unfold, how the future is going to unfold. And those things that you're predicting. You can track, and you can track them back to the decision and you can do it pretty darn fast, mind you. I mean, think about being in 2021. There were companies that were raising in A six months after seed. Today, it's a little more 16 months-ish. But even so, let's just say that that was the only thing that you decided to do. I'm going to forecast the probability that this company's going to fund at Series A. And then obviously, those companies start to fund or not fund at Series A and you're finding that out in 16 months. Here's my question for you, Lenny. Is 16 months shorter than 10 years?
你必須了解的是，你正在對世界如何發展、未來如何展開做出預測。而那些你正在預測的事情，你可以追蹤，並且可以快速地追溯到決策上。我是說，想想 2021 年，有些公司在種子輪後六個月就進行 A 輪融資。今天，大約是 16 個月左右。但即便如此，假設這是你唯一決定要做的事情。我將預測這家公司在 A 輪融資的概率。然後顯然，那些公司開始在 A 輪融資或不融資，你在 16 個月內就能知道結果。Lenny，我有個問題要問你，16 個月比 10 年短嗎？

(00:51:38):
So it's probably why he said to ask me that question because I just really do. I mean, I have a very strong opinion about this. The feedback loop is as long as you choose it to be. And if I take that back to some of the things that I heard early on when I was talking to people, what I would say is that I think that there is a certain amount of psychological safety in allowing the feedback loop to stay long because really of two main factors. One is that, look, if I was early into Uber and now I'm a celebrity investor or something, I don't really want to know if I'm good or not. Do I? Right? I don't really want the world to know that. I mean, if I'm good, that's great. But it feels like they already believe that I'm good because I happen to be early into Uber.
所以這可能就是為什麼他說要問我這個問題，因為我真的有很強烈的看法。反饋循環的長短取決於你自己。如果我回想起早期和人們交談時聽到的一些事情，我會說，允許反饋循環保持較長時間在心理上有一定的安全感，主要有兩個原因。首先，如果我早期投資了 Uber，現在成為了一個名人投資者，我真的不想知道自己是否真的很厲害，對吧？我不想讓世界知道這一點。我是說，如果我很厲害，那當然很好。但感覺他們已經相信我很厲害了，因為我早期投資了 Uber。

(00:52:47):
So since people already think I'm good, I'm just losing to that decision, psychologically speaking. Not investment quality speaking, but psychologically speaking. Okay, so here's the problem though. Why was I early into Uber? Did I have an insight into a real pain point in a developing market, blah, blah, blah? Or did my buddy start Uber and I was like, "Sure, I'll give you some money?" Right? I mean, obviously, I'm talking about the extremes here, but we don't actually know what the decision quality was, right? All we know is that you had a good result and given that you had a good result and people think very highly of you, what are you going to gain? Right? So it's so nice to just let that feedback loop sit there and allow people to have the opinion of you. That is really nice, feels good, right? And not actually find out the answer because why would I want to?
所以，既然人們已經認為我很厲害，從心理學角度來說，我只是屈服於這個決定。不是從投資質量的角度，而是從心理學的角度。好，問題來了。為什麼我會早早投資 Uber？是因為我對發展中市場的真實痛點有洞察力，等等嗎？還是因為我的朋友創辦了 Uber，我就說，「好吧，我給你一些錢」？對吧？我的意思是，顯然我在談論極端情況，但我們實際上並不知道決策的質量如何，對吧？我們只知道你有了一個好的結果，既然你有了一個好的結果，人們對你評價很高，你會得到什麼好處？對吧？所以，讓這個反饋循環存在，讓人們對你有這樣的看法，真的很不錯，感覺很好，對吧？而且實際上不去找出答案，因為我為什麼要這麼做呢？

(00:53:51):
Unless you're really super focused on decision quality, then you would want to do that. So that's part of the psychological safety. And what that goes to is, the real core of it is that it's very, very difficult for human beings to deal with feeling wrong in the moment, even if it helps them in the long run. It's just hard. And the tighter the feedback loop, the more that you risk finding out you are wrong in the short run. Now that helps you to learn and improve your decision-making if you're focused on it and you're good at it, right? That's going to help you. And then in the long run, you're actually going to do better. But human beings are notoriously good at trading off the long run just to feel good in the short run. That's why we're all eating chocolate and cupcakes and stuff that we know is bad for us because it feels good.
除非你真的非常專注於決策質量，否則你會想要這樣做。這是心理安全的一部分。其核心在於，人類在當下感覺自己錯了是非常困難的，即使這在長期來看對他們有幫助。這真的很難。而反饋迴路越緊密，你在短期內發現自己錯誤的風險就越大。現在，如果你專注於學習並且擅長這樣做，這會幫助你學習和改進你的決策能力，對吧？這會幫助你。然後從長遠來看，你實際上會做得更好。但人類在短期內感覺良好而犧牲長期利益方面是出了名的擅長。這就是為什麼我們都在吃巧克力和杯子蛋糕等我們知道對我們有害的東西，因為這感覺很好。

(00:54:45):
And so much of our decision-making is trying to advance this positive self-narrative and the idea that yeah, we're going to have a more positive narrative ourselves in five years if we do some stuff. Most of us are like, "No, I don't really want to make that trade. I'd rather just feel good now", and I can use the fact that we are living in power law, under the influence of power law. I can use that to just confirm a lot of things that I wish to believe that are true of me. And if you take that away from me and you take the uncertainty away from me, it's going to be really hard. And I will tell you that's what I love about both Renegade and First Round is that they're just like, I want to know. It would be such a horror for me to think that I was making good decisions when I actually wasn't. And that's what really matters to me and I think that it's just so special.
我們的許多決策都是在試圖推進這種積極的自我敘述，以及這種想法，即如果我們做一些事情，五年後我們會有一個更積極的敘述。我們大多數人會說：「不，我真的不想做那種交換。我寧願現在就感覺良好」，而且我可以利用我們生活在冪律之下這個事實，來確認很多我希望相信的關於我自己的事情。如果你把這些從我身上拿走，把不確定性拿走，這會非常困難。我會告訴你，這就是我喜歡 Renegade 和 First Round 的原因，因為他們就像是，我想知道。對我來說，認為自己在做出好決策時實際上並不是，這將是一種恐怖。而這對我來說才是真正重要的，我認為這真的很特別。

Lenny Rachitsky (00:55:47):
倫尼·拉奇茨基 (00:55:47):
Hearing all this makes me want to be an LP in First Round. Not that they would let me in there, but just knowing that you're in there futzing with everything and the way they're thinking is really inspiring. I'm curious if you could share an example of anything that they tweaked as a result of this analysis that you did and how they evaluate.
聽到這些讓我想成為 First Round 的 LP。雖然他們不會讓我進去，但只知道你在那裡搞東搞西，並且他們的思維方式真的很鼓舞人心。我很好奇你是否可以分享一個例子，說明他們因為你做的這個分析而調整了什麼，以及他們如何評估。

Annie Duke (00:56:05): 安妮·杜克（00:56:05）：
Well, first of all, let me just say they didn't really record a lot about their decisions when I first came in. They voted and they had a record of the vote so they knew who said yes and who said no, but they didn't have a lot of other information. So the first thing that happened was just that. Okay, what do we really think is the way that you would model whether you should invest or not invest in? Very broadly, you would say you're reading the market, the team, the founder, the product broadly. We're going to make sure that those opinions aren't just like it's good or it's bad, but it's on a scale of one to seven so that we can actually get some precision and some spread among the partners in terms of, say, strength of market. We're going to make sure that we have shared definitions of those things, which you'd be surprised people don't.
首先，讓我說明一下，當我剛加入時，他們並沒有詳細記錄他們的決策。他們會投票，並且有投票記錄，所以他們知道誰投了贊成票，誰投了反對票，但除此之外並沒有太多其他資訊。所以，第一件發生的事情就是這樣。好吧，我們真的認為應該如何建模來決定是否應該投資？大致上，你會說你在評估市場、團隊、創辦人、產品。我們要確保這些意見不僅僅是好或壞，而是用一到七的尺度來評估，這樣我們才能在合夥人之間對市場的強度有更精確和更廣泛的分佈。我們還要確保我們對這些事物有共同的定義，這點你可能會驚訝，很多人並沒有。

(00:57:00):
So when I am thinking about market and market quality, I might be thinking about something very different than you. So we want to make sure that we have a shared definition of that, and that's reflected in something that we would call mediating judgments, which are judgments that you make related to market prior to actually judging what you think of the market in general. So you could think of something like competitive landscape, so you would judge that. So that turns into what these mediating judgments are, which is basically an implied definition.
所以當我在思考市場和市場質量時，我可能在想的東西和你很不一樣。因此，我們要確保我們有一個共同的定義，這反映在我們所稱的中介判斷中，即在實際評價市場整體之前，你對市場所做的判斷。你可以想像競爭格局，你會對其進行評價。這就變成了這些中介判斷，基本上是一個隱含的定義。

(00:57:30):
So you create that, and then you also think about what are the forecasts that are important. One thing that you already know is you're going to forecast the probability the company funds at Series A. So that was a huge change, just a very different way of making decisions. What we've been able to do with that now because I've been there for five years is we now can actually look, say the partnership as a whole, and look at these ratings that they're making of the component pieces of parts of how they're modeling, like what makes a good investment, these forecasts that they're making.
所以你創建了那個，然後你也要考慮哪些預測是重要的。有一件事你已經知道了，就是你要預測公司在 A 輪融資的概率。所以這是一個巨大的變化，是一種非常不同的決策方式。因為我已經在那裡工作了五年，我們現在能夠做的是，實際上可以看整個合夥關係，並查看他們對組成部分的評級，這些評級是他們在建模時做出的，比如什麼是好的投資，這些預測是他們做出的。

(00:58:03):
And we actually know how these companies have now unfolded. So in the simplest sense, we know whether hundreds of companies have funded at Series A or not. And now, we can actually look and say, how good are the partners at actually forecasting this thing? Right? Are they random? Are they better than random? And we actually know that and we can feed that back to them so that they can understand their own accuracy around these pieces of the decision because the fact is that whatever a seed investor says to you, whether that company is going to fund at Series A is part of their decision. It's included in the decision. So they're making that forecast whether they make the forecast explicitly or not. So what we're saying over at First Round is let's make it explicit because you're doing it implicitly anyway, and then we can actually start to look at your accuracy.
我們其實知道這些公司現在的發展情況。簡單來說，我們知道數百家公司是否獲得了 A 輪融資。現在，我們可以實際上查看並評估合作夥伴在預測這方面的準確性。對吧？他們是隨機的嗎？還是比隨機更好？我們實際上知道這些，並且可以將這些信息反饋給他們，讓他們了解自己在這些決策方面的準確性。事實是，無論種子投資者對你說什麼，該公司是否會在 A 輪融資中獲得資金是他們決策的一部分。這包含在決策中。所以他們在做這個預測，不管他們是否明確地做出這個預測。因此，我們在 First Round 所說的是，讓我們把它明確化，因為你無論如何都在隱含地做這件事，然後我們可以實際上開始查看你的準確性。

(00:58:58):
We can now feed that back to you and let you know how accurate you are, which will then help you to become more accurate, right? We can also look, because we know in any given vintage what are the best companies or what are the worst companies. Remember, we're having people do these ratings on a scale of one to seven of say the quality of the market, and we can look across the partnership and say, "Look, Lenny, when you think the market is great, how does that map onto how well the company is doing in the future? When you say the market is terrible, how well does that map onto how that company is doing, how that company ends up doing in the future?" And I can come to you and I can say, "Oh, Lenny, by the way, your judgments about market are amazing."
我們現在可以將這些反饋給你，讓你知道你的準確度如何，這將幫助你變得更加準確，對吧？我們還可以查看，因為我們知道在任何特定年份中哪些是最好的公司或最差的公司。記住，我們讓人們對市場質量進行一到七的評分，我們可以在合作夥伴之間進行比較，然後說：「看，Lenny，當你認為市場很好的時候，這與公司未來的表現有什麼關聯？當你說市場很糟糕時，這與公司當前的表現以及未來的結果有什麼關聯？」然後我可以對你說：「哦，Lenny，順便說一下，你對市場的判斷非常了不起。」

(00:59:47):
You know a good market when you see one and it's really mapping on in a great way onto how that company unfolds, but you're not so great with founder, or maybe you're great across the board, or maybe you're not. So we can now start to give people, we can give the partners insight into their own decision-making, not only to allow them to improve the decision-making, but also to allow them to understand not to over index on certain things that maybe aren't as predictive as an example for them. We also can change the rubric based on the evidence now.
當你看到一個好的市場時，你會知道它是好的，而且它與那家公司發展的方式非常契合，但你可能不太擅長處理創始人，或者你可能在各方面都很擅長，或者你可能不擅長。所以我們現在可以開始給人們提供一些見解，讓合夥人了解他們自己的決策過程，不僅能幫助他們改進決策，還能讓他們明白不要過度依賴某些可能對他們來說並不那麼具有預測性的因素。我們也可以根據現在的證據來改變評估標準。

(01:00:22):
So the first version of the rubric is always taking the intuitions of the partners and making those explicit, but then we can start to loop those back together and understand, well, maybe this thing that the partners thought was important actually isn't predictive across any partner. For example, we can start to develop the rubric based on the data. These were all things that weren't possible because prior to that, if I had come in and said, "Well, let's look at decision quality", how would I do that? I mean, I have no idea why people were... I just know whether they said yes or no. And so it's very difficult to them to start to do some really serious refining of the decisions if I don't have that information.
所以，第一版的評分標準總是基於合夥人的直覺，並將這些直覺明確化，但接下來我們可以開始將這些直覺回饋並理解，也許合夥人認為重要的事情其實對任何合夥人都沒有預測性。例如，我們可以開始根據數據來制定評分標準。這些都是以前無法做到的事情，因為在那之前，如果我進來說，「那麼，讓我們看看決策質量」，我該怎麼做呢？我的意思是，我完全不知道人們為什麼會這樣做……我只知道他們是否說了「是」或「否」。因此，如果我沒有這些信息，就很難開始對決策進行一些真正嚴格的改進。

Lenny Rachitsky (01:01:08):
I desperately want to know which partner it makes the best decisions. I know you're not going to share them.
我非常想知道哪個合夥人做出最好的決定。我知道你不會分享他們。

Annie Duke (01:01:13):
No, I'm not going to. The partnership as a whole is excellent as we know. And this is what I will tell you is that all partners have strengths and all partners have weaknesses, and they're not perfectly overlapping, which is wonderful, right? I mean, that's one of the things. It's like what's really wonderful, and I think it shows the power of why would you have more than one person having input into a decision, is some people are very strong on rating a particular aspect of market, or some people are very strong on rating a particular aspect of the founder. There's overlap and then there's things where Todd is uniquely great at something, or Josh is uniquely great at something, or Bill is uniquely great at something. So that's a wonderful thing about it is that everybody has strengths and everybody has weaknesses and they're not perfectly overlapping. So this is where you can see getting that spread of opinions and really understanding, breaking that decision down into its component parts really shows you the value of diverse opinions as input into a decision.
不，我不會這麼做。整體來說，這個合作夥伴關係非常出色，這是我們都知道的。我想告訴你的是，每個夥伴都有優勢和劣勢，而且這些優劣勢並不完全重疊，這是很棒的，對吧？我的意思是，這就是其中一個很棒的地方。我認為這顯示了為什麼會有多個人參與決策的力量，有些人在評估市場的某個特定方面非常強，有些人在評估創始人的某個特定方面非常強。有重疊的部分，也有一些地方是托德特別擅長的，或者喬希特別擅長的，或者比爾特別擅長的。所以這是它的美妙之處，每個人都有優勢和劣勢，而且這些優劣勢並不完全重疊。因此，這就是你可以看到獲得多種意見並真正理解、將決策分解成其組成部分的價值所在，這真正顯示了多樣化意見作為決策輸入的價值。

Lenny Rachitsky (01:02:21):
I could talk about this thread forever. Maybe let me just ask one more question just because I'm super curious. Is there anything surprising that stands out, that came out of this analysis so far? Just like, oh wow, maybe market isn't as important as we thought or this person is amazing at-
我可以永遠談論這個話題。也許讓我再問一個問題，因為我真的很好奇。到目前為止，這個分析中有什麼令人驚訝的地方嗎？就像，哦，市場可能不像我們想像的那麼重要，或者這個人非常厲害-

Annie Duke (01:02:34): 安妮·杜克（01:02:34）：
Yes. So I think just generally speaking, when you're creating the initial decision rubric, there are things that people are really pounding the table about that they think is especially important in making a decision. And one of the things that we found is that sometimes their intuition was absolutely right. The thing that they were pounding the table about is incredibly predictive, not just for them but other partners. But sometimes it's not at all predictive, and these are equal table-pounding situations. So let's say you're pounding the table about something, sometimes the thing you're pounding at the table about is predictive for you and for all the other partners that it's actually quite predictive about how the company does. But sometimes when you're really pounding the table about something, it's not just that it's not predictive for the other partners, it's not predictive for you.
好的。一般來說，當你在創建初始決策標準時，有些事情是人們非常堅持認為在做決策時特別重要的。我們發現，有時他們的直覺是完全正確的，他們堅持的事情不僅對他們自己，而且對其他合夥人來說都是非常有預測性的。但有時這些事情根本沒有預測性，這些情況都是同樣堅持的。所以，假設你在堅持某件事，有時你堅持的事情對你和所有其他合夥人來說確實是很有預測性的，能夠預測公司的表現。但有時當你非常堅持某件事時，不僅對其他合夥人沒有預測性，對你自己也沒有預測性。

(01:03:33):
And I think that what's really important to understand about this, and this is why it's so incredibly necessary in improving decision quality to take what's implicit and make it explicit, is that our intuition is sometimes right and sometimes wrong. It's not that intuition is crap and your intuition is just completely wrong, I mean obviously, that can't be true. We would die, right? So your intuition is sometimes right, but it's also sometimes wrong. And if you don't make it explicit, then you don't get to find out when it's wrong.
我認為，理解這一點非常重要，這也是為什麼在提高決策質量時，將隱含的東西顯性化是如此必要的原因。我們的直覺有時是對的，有時是錯的。並不是說直覺完全沒用，或者你的直覺完全錯誤，顯然這不可能是真的，否則我們早就完蛋了。所以你的直覺有時是對的，但也有時是錯的。如果你不把它顯性化，那麼你就無法發現它何時是錯的。

(01:04:07):
You don't get to find out when it's off base, and that's a disaster. So that's the thing that I think is really interesting is that you have equal vehement and confidence that this particular factor is really important. And sometimes it is and sometimes it isn't. And it's so surprising because we're talking about people who are true experts who are great, and I think that we all just have intuition about intuition, right? You just intuit that if they're so amazing, clearly their intuition about what's important would have to be good, but not necessarily. That's the thing, not necessarily. Yeah, I think that was probably the most exciting thing.
你無法知道它何時偏離正軌，這是一場災難。所以我認為真正有趣的是，你對這個特定因素的重視程度和信心是相等的。有時候它確實很重要，有時候則不然。這非常令人驚訝，因為我們談論的是那些真正的專家，他們很出色，我認為我們都有關於直覺的直覺，對吧？你只是直覺地認為，如果他們如此出色，顯然他們對重要事物的直覺也會很好，但事實並非如此。這就是問題所在，不一定是這樣。是的，我認為這可能是最令人興奮的事情。

Lenny Rachitsky (01:04:56):
Keeping it mysterious, but I still appreciate you sharing.
保持神秘，但我仍然感謝你分享。

Annie Duke (01:04:59): 安妮·杜克（01:04:59）：
Well, I have to keep it mysterious.
嗯，我必須保持神秘。

Lenny Rachitsky (01:05:02):
Yes, I understand. 好的，我明白。

Annie Duke (01:05:03): 安妮·杜克（01:05:03）：
I can't give away the trade secrets.
我不能洩露商業機密。

Lenny Rachitsky (01:05:06):
I wanted to touch on a different framework that I've heard a lot of companies actually using, and something that comes up in product a lot is this idea of pre-mortems, which is essentially think ahead of time what might go wrong. Can you talk about this? Because I think that it's something that's easy to implement, really powerful, and a lot of people are actually doing this.
我想談談另一個我聽說很多公司實際上在使用的框架，並且在產品中經常出現的一個概念，就是這個預先檢討的想法，基本上是提前思考可能會出現的問題。你能談談這個嗎？因為我認為這很容易實施，非常強大，而且很多人實際上都在這樣做。

Annie Duke (01:05:25): 安妮·杜克（01:05:25）：
Yes, okay. So a pre-mortem is great, but only if you attach a pre-commitment with it. So I just want to be super clear about that. What you find with pre-mortems, this actually work. I actually did this work with Maurice Schweitzer and Linnea Gandhi, who are both at Penn. Then when you have people do pre-mortems, it generally doesn't actually change their plan very much or it changed their behavior. So I think that we have the feeling that if you do a pre-mortem and you think what are the ways that things might go wrong, that that's going to change your plan, but probably not unless you're specifically using it for that purpose and you say, "Okay, we're going to do this, but let's think about how we might change our plan in light of this information." But I think what's actually more important than that is what a pre-mortem allows you to do is to set up kill criteria.
好的，沒錯。事前檢討是很棒的，但只有在你附加一個事前承諾時才有效。所以我想要非常清楚地說明這一點。你會發現事前檢討確實有效。我實際上和 Maurice Schweitzer 和 Linnea Gandhi 一起做過這項工作，他們都在賓州大學。當你讓人們做事前檢討時，通常並不會真正改變他們的計劃或行為。所以我認為我們有一種感覺，如果你做了事前檢討並思考事情可能出錯的方式，那會改變你的計劃，但實際上可能不會，除非你特意為此目的使用它，並且你說，「好，我們要做這個，但讓我們思考一下根據這些信息我們可能會如何改變計劃。」但我認為比這更重要的是，事前檢討讓你能夠設立終止標準。

(01:06:18):
So kill criteria are just a set of signals that you might see that would tell you that it's time to pivot or stop because once we actually launch something, we're very, very slow to decide to quit. I'm sure that everybody has felt that way before. Things go on way too long, even like they're over budget and you've blown the timeline. And when you finally shut it down, you realize you should have done it a lot earlier. And this is true across the board because of a variety of biases. The most well-known, and probably the biggest influence is something called sunk cost, which is that feeling, but then I'll have wasted all the time and effort that I've put into this already. So it's taking into account what you've put in in the past and deciding whether to continue on in the future. So what we want to do is actually just get better at that thing. So understanding that when you've gotten to the pre-mortem process, you probably are going to launch it, like it's probably going to be the case.
所以「終止標準」只是一組信號，告訴你是時候轉向或停止，因為一旦我們實際上推出某個東西，我們在決定放棄時會非常非常慢。我相信每個人都曾有過這樣的感覺。事情拖得太久，甚至超出預算，時間線也被打亂。當你最終關閉它時，你會意識到你應該早就這麼做了。這在各個方面都是如此，因為存在各種偏見。最著名、可能影響最大的就是所謂的「沉沒成本」，這種感覺是，我已經投入了這麼多時間和精力，如果現在放棄，那麼這些都白費了。所以這是考慮到你過去的投入，並決定未來是否繼續。因此，我們想要做的其實是更好地掌握這一點。理解當你進入「事前驗屍」過程時，你可能會推出它，這很可能會成為現實。

(01:07:22):
Use the pre-mortem to set up kill criteria. So I'll give you an example from a sales team that I worked with. Basically, I sent them out a prompt, all the IC is a prompt that was, imagine that you got a lead through an RFP or RFI and you worked on it for six months. And now, it's six months later and the deal is dead. Looking back, you realized they were early signals that that was going to be the case, what were they. So this is a pre-mortem. What are the things that you saw that would tell you that this was going to go south? And they came up with all sorts of ideas. Notice this doesn't mean they're not going to start off pursuing the lead, right? But they saw, they came up with all sorts of signals. So I'll just give you three of them.
使用事前檢討來設置終止標準。我來舉個我曾經合作過的一個銷售團隊的例子。基本上，我給他們發了一個提示，所有的個人貢獻者（IC）都收到了一個提示，假設你通過 RFP 或 RFI 獲得了一個潛在客戶，並且你已經為此工作了六個月。現在，六個月過去了，這筆交易失敗了。回顧過去，你意識到早期就有一些信號表明會是這樣的情況，那些信號是什麼？這就是事前檢討。你看到了哪些跡象告訴你這件事會走向失敗？他們提出了各種各樣的想法。注意，這並不意味著他們不會開始追求這個潛在客戶，對吧？但他們看到了，並提出了各種各樣的信號。所以我只給你三個例子。

(01:08:12):
The RFP RFI was clearly written with a competitor in mind, so they felt that was a very bad signal that was probably going to go badly. Another one was the customer didn't want to demo, they only wanted to talk about price. Obviously, that's quite bad. And another one was after the first few meetings, they couldn't get a decision maker in the room, right? So it was a much longer list than this, but those are three. So for each of those, that now becomes a kill criteria if I see this thing and now you attach an action with it. So in the case of price, they actually just said, "We should kill it." If they literally don't want to demo and they're only asking about price, they're just trying to beat up somebody else on price like we're a box-checking exercise. So there, they just said, "We're going to kill it. We're not going to pursue the deal anymore."
這份 RFP RFI 顯然是針對競爭對手寫的，所以他們覺得這是一個非常不好的信號，可能會發展得很糟糕。另一個例子是客戶不想看演示，只想談價格。顯然，這非常糟糕。還有一個例子是，在前幾次會議之後，他們無法讓決策者參加會議。所以這是一個更長的清單，但這是其中的三個例子。因此，對於每一個這樣的情況，現在都成為一個終止標準，如果我看到這種情況，現在你需要附上一個行動方案。所以在價格的情況下，他們實際上只是說，「我們應該終止它。」如果他們真的不想看演示，只是問價格，他們只是想在價格上打壓別人，就像我們是一個打勾的練習一樣。所以在這裡，他們只是說，「我們要終止它。我們不再追求這筆交易。」

(01:08:59):
So this is great because salespeople will pursue deals forever and leadership is like, "Well, why did you stop pursuing that deal?" And they get in trouble for it. So this is going to help with that problem, right? In the case of the RFP RFI was written with a competitor in mind, they have an action associated with that as well, which is ask them directly if they're working with a competitor and how far down the road they are, depending on the answer you would kill or pursue. In the case of we couldn't get a decision maker in the room, offer up executive alignment at the next meeting. And if they say sure, great. And if they say no, kill.
這很棒，因為銷售人員會無限期地追求交易，而領導層則會問：「為什麼你停止追求那筆交易？」然後他們會因此而受到責備。所以這將有助於解決這個問題，對吧？如果 RFP（徵求建議書）或 RFI（徵求資訊書）是針對競爭對手而寫的，他們也有相應的行動，那就是直接問他們是否與競爭對手合作，以及進展到什麼程度，根據答案決定是放棄還是繼續追求。如果我們無法讓決策者參加會議，那麼在下一次會議中提供高層對接。如果他們同意，那就很好。如果他們拒絕，那就放棄。

(01:09:35):
So that's actually what I feel is the best use of a pre-mortem is to say, I'm going to try to figure out what those signals are along the way that things are going badly. And now instead of just hoping that when I see those signals, I actually act rationally, which is a hope that it will not come true. That's why there's many people who climb Everest in the middle of a blizzard, even though they shouldn't be doing that. Use the pre-mortem to now create structure around those signals that you've spotted and commit to actions that you're going to take if you see those signals. And I think that's the best use of a pre-mortem.
所以，這其實是我認為預先檢討（pre-mortem）最好的用途，就是說，我要嘗試找出那些事情進展不順利的信號。現在，不僅僅是希望當我看到那些信號時，我能夠理性行事，這種希望往往是不會實現的。這就是為什麼有很多人在暴風雪中攀登珠穆朗瑪峰，即使他們不應該這樣做。利用預先檢討來圍繞你發現的那些信號建立結構，並承諾如果你看到那些信號時要採取的行動。我認為這是預先檢討的最佳用途。

Lenny Rachitsky (01:10:15):
倫尼·拉奇茨基 (01:10:15)：
That's really helpful. And it's interesting how many of your examples come back to just of a framework that you often talk about, which is make it explicit what is implicit. There's another example that the First Round example is a great example of that where it's just, here's all our assumptions, they just make them actually explicit and just shows how much [inaudible 01:10:32].
這真的很有幫助。而且有趣的是，你的許多例子都回到你經常談論的一個框架，那就是將隱含的東西顯性化。另一個例子是 First Round 的例子，這是一個很好的例子，他們只是把所有的假設都顯性化，這顯示了多少 [聽不清 01:10:32]。

Annie Duke (01:10:32): 安妮·杜克 (01:10:32):
Right, then you can examine them. You can examine them, people can discuss them, you can figure out if they're wrong or right or whatever. It's like, I want to be very clear, I'm not anti your gut or your intuition. I think it's probably sometimes pretty good. I just want you to make it explicit, that's all.
對，然後你可以檢查它們。你可以檢查它們，人們可以討論它們，你可以弄清楚它們是錯的還是對的或其他什麼。我想非常清楚地說，我不是反對你的直覺或直感。我認為它有時可能相當不錯。我只是希望你把它顯性化，就這樣。

Lenny Rachitsky (01:10:55):
Okay. So we didn't have time to get into quitting, which is your more recent book. Maybe we'll do a follow-up episode specifically thinking about quitting, but let me just ask-
好的。我們沒有時間討論到你的新書《放棄》。也許我們會做一個後續的節目專門討論放棄，但讓我先問一下——

Annie Duke (01:11:03): 安妮·杜克 (01:11:03):
It's my fault because my answers are long.
這是我的錯，因為我的回答太長了。

Lenny Rachitsky (01:11:05):
Lenny Rachitsky（01:11:05）：
It's my fault. 這是我的錯。

Annie Duke (01:11:06): 安妮·杜克 (01:11:06):
I apologize to everybody.
我向大家道歉。

Lenny Rachitsky (01:11:06):
萊尼·拉奇茨基 (01:11:06):
No apology is necessary. We'll have plenty of time in the future, hopefully. Well, let me just ask one question. I found this one quote from you where you said you should assume that if you're thinking about quitting, it's already probably past the time that you should have quit. Do you still believe that? Is that generally a good rule of thumb? And just any takeaway, tip, lesson on quitting as our one question on quitting?
不需要道歉。我們未來應該會有很多時間。那麼，讓我問一個問題。我發現你曾說過一句話，如果你在考慮放棄，那可能已經是你應該放棄的時候了。你還相信這句話嗎？這通常是一個好的經驗法則嗎？關於放棄，有沒有什麼心得、建議或教訓可以分享？這是我們關於放棄的唯一問題。

Annie Duke (01:11:29): 安妮·杜克（01:11:29）：
So the data is pretty strong that by the time you quit, it's probably long after the [inaudible 01:11:37]. And it's really just because, look, when we start things, we're starting things under difficult circumstances, which come from the uncertainty of the decision to start something. So when we start something, luck is going to have an influence on the outcome, which obviously, we have no control over because luck isn't in our control. And then there's also hidden information. So what happens is that after the fact, we know that we're going to learn new information and it can make it very hard to start things because we want to be more sure than we actually need to be. It's why Bezos has the 70% role to try to roll people back and be willing to accept that uncertainty in the starting decision. Now the good news is that when you learn that new stuff and the new stuff that you learn is, "Ooh, if I had known this, I wouldn't have started it", you have the option to quit generally.
所以數據顯示，當你決定放棄時，可能已經是很久之後的事了。這主要是因為，當我們開始做某件事時，通常是在困難的情況下開始的，這些困難來自於決定開始某件事的不確定性。因此，當我們開始某件事時，運氣會對結果產生影響，而顯然，我們無法控制運氣。還有一些隱藏的信息。所以，事後我們會學到新的信息，這會使得開始某件事變得非常困難，因為我們希望比實際需要的更有把握。這就是為什麼貝佐斯有 70%的規則，試圖讓人們接受開始決策中的不確定性。好消息是，當你學到新信息時，如果你發現「哦，如果我早知道這些，我就不會開始了」，你通常還是有選擇放棄的機會。

(01:12:28):
So that's the good news. The bad news is that the same difficulties that apply to the decision to start apply to the decision to stop. In other words, we're making that decision under uncertainty as well. So we're not going to know for sure whether it would've turned out well or poorly unless we continue to do the thing that we already started. And we don't like to walk away from things unless we know for sure. So as Richard Thaler put it, most people won't quit until it actually isn't a decision. In other words, the whole thing is blown up, the startup has no money, or you're up on Everest and the blizzard is literally upon you and you're stuck in it. Or I think as he said, until you've fallen in the crevasse already, then you'll make the decision to quit because then you know how it was really bad. So people generally, for example, don't quit their jobs until they feel they have no other choice or relationships or projects or products that they're developing, it all applies because we want to know for sure.
所以這是好消息。壞消息是，適用於開始決策的困難同樣適用於停止決策。換句話說，我們也是在不確定的情況下做出這個決定。因此，除非我們繼續做我們已經開始的事情，否則我們無法確定結果是好是壞。而且我們不喜歡在沒有確定的情況下放棄事情。正如理查德·塞勒所說，大多數人不會在事情還有選擇的時候放棄。換句話說，整件事已經崩潰了，創業公司沒有錢了，或者你在珠穆朗瑪峰上，暴風雪真的來了，你被困在其中。或者我認為他說的是，直到你已經掉進裂縫裡，你才會決定放棄，因為那時你知道情況真的很糟糕。所以一般來說，人們不會辭掉工作，除非他們覺得別無選擇，這同樣適用於他們的關係、項目或正在開發的產品，因為我們想要確定。

(01:13:35):
And then on top of that is the fact that there is this issue of sunk cost, which is when we walk away, we feel like we'll have wasted everything that we've already put into what we're doing. But of course, waste is a prospective problem, not a retrospective one, even if we treat it like a retrospective one because it's the prospective one. Well, if you wouldn't start this today, then that means that everything that you're putting into this going forward is the actual waste, right? But we do that all the time. We go forward with things that we ought not to be going forward with because we're trying to protect the resources that we've already sunk into it in the past. Then there's other issues that have to do with, for example, endowment, the ownership over the things that we've built. This is particularly bad in product because we're building things.
再加上還有沉沒成本的問題，當我們放棄時，會覺得已經投入的一切都浪費了。但當然，浪費是一個前瞻性問題，而不是回顧性問題，即使我們把它當作回顧性問題來對待，因為它其實是前瞻性的。那麼，如果你今天不會開始這件事，那麼你未來投入的所有東西才是真正的浪費，對吧？但我們經常這樣做。我們繼續推進那些不應該推進的事情，因為我們試圖保護過去已經投入的資源。然後還有其他問題，例如所有權，我們對已經建立的東西的所有權。這在產品開發中特別糟糕，因為我們正在構建東西。

(01:14:18):
And once we build things, we own them. And once we own them, we don't want to give them up and we actually value them more highly than identical things that we don't own. And then there's issues of internal and external validity, which is really just a fancy way to say your identity. How do other people view you? How do you view yourself? Do you feel like you failed? And what that means is that by the time, there's so many biases against stopping that by the time you're actually even thinking about quitting, it's probably already past the time that you ought to have quit. But we'll still continue on until we know for sure we didn't have any choice because here's the thing. When you walk away from something and someone's like, "Hey, why'd you stop that?" And you're like, "Oh, I had no choice" and you tell them everything that went wrong and, nobody's going to question you.
一旦我們建造了某些東西，我們就擁有它們。一旦我們擁有它們，我們就不想放棄它們，並且我們實際上會比那些我們不擁有的相同東西更重視它們。然後還有內部和外部有效性的問題，這其實只是說你的身份。別人怎麼看你？你怎麼看自己？你覺得自己失敗了嗎？這意味著，當你真正開始考慮放棄的時候，可能已經過了你應該放棄的時間。但是我們仍然會繼續，直到我們確定自己別無選擇，因為事情是這樣的。當你放棄某件事時，有人問你：「嘿，你為什麼停下來？」你會說：「哦，我別無選擇」，然後告訴他們所有出錯的事情，沒有人會質疑你。

(01:15:04):
They're going to be like, "Oh, well, it sounds like you put in your best effort." But if you walk away early, people are like, "What?" So just quickly, I'll tell you I think just one of the best stories of this that I've got in my pocket here. Let me pull it out before the end. So Stewart Butterfield creating a product called Glitch, and Glitch is a massive multiplayer online world-building cooperative game. Releases it, this is in the aughts, and it's like a huge hit with the critics. It's Monty Python meets Dr. Seuss, it's an incredible whatever. They're getting tons and tons of great word of mouth and PR not doing any paid marketing. They have incredible investors in Andreessen Horowitz and Accel, they have $6 million in the bank and they 5,000 have diehard users, meaning users who use the game who play over 20 hours a week.
他們會說：「哦，看起來你已經盡了最大的努力。」但如果你提早離開，人們會說：「什麼？」所以我快速告訴你，我認為這是我口袋裡最好的故事之一。讓我在結束前把它拿出來。Stewart Butterfield 創造了一個名為 Glitch 的產品，Glitch 是一個大型多人在線世界建設合作遊戲。發布的時候，這是在 2000 年代，評論家們對它讚譽有加。它就像是 Monty Python 遇上 Dr. Seuss，非常不可思議。他們通過口碑和公關獲得了大量的好評，沒有做任何付費營銷。他們有 Andreessen Horowitz 和 Accel 這些令人難以置信的投資者，銀行裡有 600 萬美元，並且有 5000 名死忠用戶，這些用戶每週玩超過 20 小時。

(01:16:04):
The issue is that customer acquisition was a beast that for every one person who was playing over 20 hours a week, there were between 95 and 99 people who came for five minutes and left. So obviously, this is a customer acquisition problem, which everybody knows. So they make an agreement in 2012 that they're going to do paid marketing, which they do for six weeks. And during that six weeks, their acquisition of new users, it's growing like six to 7% week over week, which is amazing. And at the end of that six weeks, this is November of 2012, at the end of that six weeks, that Monday morning, Stewart Butterfield writes a note to his investors and co-founders and says, "I woke up this morning with the dead certainty that Glitch was over."
問題在於，客戶獲取是一個難題，每一個每週玩超過 20 小時的人，對應有 95 到 99 個人只玩了五分鐘就離開了。所以顯然，這是一個客戶獲取的問題，這是大家都知道的。因此，他們在 2012 年達成協議，決定進行付費行銷，他們進行了六週的付費行銷。在這六週期間，他們的新用戶獲取量每週增長 6%到 7%，這是非常驚人的。在這六週結束時，也就是 2012 年 11 月，六週結束的那個星期一早上，Stewart Butterfield 寫了一封信給他的投資者和共同創辦人，說道：「我今天早上醒來，確信 Glitch 已經結束了。」

(01:16:58):
Now notice nobody would do this in this case, right? But this is what happened is that the issue is that you really have to see is this worthwhile or not? Would I start this today? That's a forecast of the future, right? And what he did was some back-of-the-envelope math, and he said, "Look, if we continue to acquire customers at the weight that we've been acquiring them at the cost that we've been acquiring them, it's going to be 31 weeks until we break even." But that's an absurd assumption because customer acquisition costs, it's going to go up. Cap has to rise because we're going to saturate the core gaming market, so it's got to rise. So what he realized at that point was that this was not a venture scale business, and he was in this for a venture scale business. So even though nobody else saw that he was supposed to shut it down, he saw he was supposed to shut it down.
現在注意，沒有人會在這種情況下這樣做，對吧？但實際上發生的情況是，你真的必須看看這是否值得。今天我會開始這個嗎？這是一種對未來的預測，對吧？他做了一些簡單的計算，說：「看，如果我們以目前的速度和成本繼續獲取客戶，31 週後我們就能收支平衡。」但這是一個荒謬的假設，因為客戶獲取成本會上升。獲客成本必須上升，因為我們會飽和核心遊戲市場，所以它必須上升。所以他在那時意識到，這不是一個有風險投資規模的業務，而他是為了風險投資規模的業務而來的。因此，儘管沒有人認為他應該關閉它，但他認為他應該關閉它。

(01:17:45):
And not only that, he saw that he was supposed to shut it down for his employees who were working for equity, and he had now realized that the equity wasn't worth their time and that it wasn't fair to them to keep going with it. So he shuts it down. Obviously, that feels... Who does that? Right? And he will actually tell you that he knows he should have shut it down before the marketing push, but he needed the marketing push to prove it to himself that he was seeing the future clearly. Now, the code to this story just quickly is two days later, he's like, "Well, I'm a startup guy. I want to start something", and he's got this internal communication tool that his team is using in order to develop this product that everybody loves. And he says, "Actually, they really like that. Maybe that should be the next product."
不僅如此，他還意識到他應該為那些為股權工作的員工關閉公司，因為他現在明白這些股權不值得他們花時間，繼續下去對他們不公平。所以他關閉了公司。顯然，這感覺……誰會這麼做？對吧？他其實會告訴你，他知道自己應該在市場推廣之前就關閉公司，但他需要市場推廣來證明自己看清了未來。現在，這個故事的結尾是兩天後，他說：「我是個創業者，我想創辦點什麼」，而他的團隊正在使用一個內部溝通工具來開發這個大家都喜歡的產品。他說：「其實，他們真的很喜歡這個工具，也許這應該成為下一個產品。」

(01:18:29):
So he goes and talks to the investors, they roll their money over into that. And that thing, which had no name at the time, now gets a name which is searchable log of all company knowledge, which is Slack. And so I think this is the important thing to realize is that we get so focused on, "But what about everything that I've put into it?" When what we forget is that when you're doing something, there's not just the cost of doing something that's not worthwhile that's direct, but there's also the cost of not being able to devote your attention to other opportunities that might be available to you. And as smart as Stewart Butterfield is, he couldn't see Slack until he quit Glitch. And that is a true cost that he would've born of continuing with Glitch. If he had continued with Glitch, Slack would not be something that we're all using today.
於是他去和投資者談話，他們將資金轉投到那個項目。而那個當時還沒有名字的項目，現在有了一個名字，叫做公司知識的可搜索日誌，也就是 Slack。所以我認為重要的是要意識到，我們常常過於專注於「但我已經投入了這麼多」，而忘記了當你在做某件事時，不僅僅是做不值得的事情的直接成本，還有無法將注意力投入到其他可能的機會上的成本。即使像 Stewart Butterfield 這樣聰明的人，也無法在他放棄 Glitch 之前看到 Slack。而這就是他繼續做 Glitch 所要承擔的真正成本。如果他繼續做 Glitch，今天我們就不會有 Slack 這個工具了。

Lenny Rachitsky (01:19:21):
倫尼·拉奇茨基（01:19:21）：
That is an incredibly beautiful way to end our conversation. I feel like I have at least a billion more questions to ask you and on the other hand, I feel like we've also helped a lot of people make much better decisions through this chat so I'm really thankful that you made time for this. Two last questions. Where can folks find you online if they want to learn about the stuff you're up to in case they want to work with you? And how can listeners be useful to you?
這是一個非常美麗的方式來結束我們的對話。我感覺我至少還有十億個問題想問你，另一方面，我也覺得通過這次聊天我們幫助了很多人做出更好的決定，所以我真的很感謝你抽出時間來參加這次對話。最後兩個問題。如果人們想了解你在做的事情並且有意與你合作，他們可以在哪裡找到你？聽眾們怎樣才能對你有所幫助？

Annie Duke (01:19:43): 安妮·杜克（01:19:43）：
You can find me at annieduke.com if you're interested in working with me. I have a Substack, Thinking in Bets. Please go check that out. I teach a class on maven.com twice a year on effective decision-making. So if you're interested in that, you can go to Maven and check it out. My next cohort at the moment is in September, although I might do one in May. I'm not sure. But in terms of people can help me, I co-founded an organization called The Alliance for Decision Education. We're trying to bring the kinds of knowledge that we have about improving human decision-making in adults to K through 12 education to make the world a better place. So I would love it if people could go look at that. If you're interested in it, get the word out.
如果你有興趣與我合作，可以在 annieduke.com 找到我。我有一個 Substack，名為 Thinking in Bets，請去看看。我每年在 maven.com 上教兩次關於有效決策的課程。如果你對此有興趣，可以去 Maven 查看。目前我的下一期課程在九月，雖然我可能會在五月再開一班，但還不確定。至於如何幫助我，我共同創立了一個名為 The Alliance for Decision Education 的組織。我們試圖將我們在改善成人決策方面的知識帶入 K-12 教育，以使世界變得更美好。所以如果大家能去看看這個組織，我會非常感激。如果你有興趣，請幫忙傳播這個訊息。

Lenny Rachitsky (01:20:26):
倫尼·拉奇茨基（01:20:26）：
Amazing. So we'll link to all those things in the show notes. Annie, again, thank you so much for being here.
太棒了。我們會在節目筆記中鏈接所有這些內容。Annie，再次感謝你來到這裡。

Annie Duke (01:20:32):
Thank you. Thank you so much, this was so fun.
謝謝。非常感謝，這真的很有趣。

Lenny Rachitsky (01:20:35):
Same. Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.
一樣。再見，大家。非常感謝您的收聽。如果您覺得這有價值，您可以在 Apple Podcasts、Spotify 或您喜愛的播客應用程式上訂閱本節目。此外，請考慮給我們評分或留下評論，這對其他聽眾找到本播客非常有幫助。您可以在 lennyspodcast.com 找到所有過去的集數或了解更多關於本節目的資訊。下集見。