Leopold Aschenbrenner, June 2024
利奧波德·阿申布倫納，2024 年 6 月
You can see the future first in San Francisco.
你可以在舊金山首先看到未來。
Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters. Every six months another zero is added to the boardroom plans. Behind the scenes, there’s a fierce scramble to secure every power contract still available for the rest of the decade, every voltage transformer that can possibly be procured. American big business is gearing up to pour trillions of dollars into a long-unseen mobilization of American industrial might. By the end of the decade, American electricity production will have grown tens of percent; from the shale fields of Pennsylvania to the solar farms of Nevada, hundreds of millions of GPUs will hum.
在過去的一年裡，城裡的話題已經從 100 億美元的計算集群轉移到 1000 億美元的集群，再到萬億美元的集群。每六個月，董事會計劃中就會多出一個零。在幕後，為了確保本世紀剩餘時間內仍可用的每一份電力合同，每一個可能獲得的電壓變壓器，正在進行激烈的爭奪。美國的大企業正準備投入數萬億美元，進行一場久違的美國工業力量動員。到本世紀末，美國的電力生產將增長數十個百分點；從賓夕法尼亞州的頁岩田到內華達州的太陽能農場，數億個 GPU 將會嗡嗡作響。
The AGI race has begun. We are building machines that can think and reason. By 2025/26, these machines will outpace many college graduates. By the end of the decade, they will be smarter than you or I; we will have superintelligence, in the true sense of the word. Along the way, national security forces not seen in half a century will be unleashed, and before long, The Project will be on. If we’re lucky, we’ll be in an all-out race with the CCP; if we’re unlucky, an all-out war.
AGI 競賽已經開始。我們正在建造能夠思考和推理的機器。到 2025/26 年，這些機器將超越許多大學畢業生。到這個十年結束時，它們將比你或我更聰明；我們將擁有真正意義上的超級智能。在此過程中，半個世紀未見的國家安全力量將被釋放，不久後，這個項目將全面展開。如果我們幸運的話，我們將與中共展開全面競賽；如果我們不幸的話，將會是一場全面戰爭。
Everyone is now talking about AI, but few have the faintest glimmer of what is about to hit them. Nvidia analysts still think 2024 might be close to the peak. Mainstream pundits are stuck on the willful blindness of “it’s just predicting the next word”. They see only hype and business-as-usual; at most they entertain another internet-scale technological change.
現在每個人都在談論人工智慧，但很少有人知道即將發生什麼。Nvidia 分析師仍然認為 2024 年可能接近高峰。主流評論家仍然停留在「它只是預測下一個詞」的故意盲目。他們只看到炒作和一如既往的商業；最多他們會考慮另一個互聯網規模的技術變革。
Before long, the world will wake up. But right now, there are perhaps a few hundred people, most of them in San Francisco and the AI labs, that have situational awareness. Through whatever peculiar forces of fate, I have found myself amongst them. A few years ago, these people were derided as crazy—but they trusted the trendlines, which allowed them to correctly predict the AI advances of the past few years. Whether these people are also right about the next few years remains to be seen. But these are very smart people—the smartest people I have ever met—and they are the ones building this technology. Perhaps they will be an odd footnote in history, or perhaps they will go down in history like Szilard and Oppenheimer and Teller. If they are seeing the future even close to correctly, we are in for a wild ride.
不久後，世界將會醒來。但現在，或許只有幾百人，大多數在舊金山和人工智慧實驗室，擁有情境意識。通過某些奇特的命運力量，我發現自己在他們之中。幾年前，這些人被嘲笑為瘋子——但他們相信趨勢線，這使他們能夠正確預測過去幾年的人工智慧進展。這些人是否也能正確預測未來幾年，還有待觀察。但這些人非常聰明——是我見過最聰明的人——而且他們是建造這項技術的人。也許他們會成為歷史上的一個奇怪註腳，或者他們會像西拉德、奧本海默和泰勒一樣名垂青史。如果他們對未來的預見即使接近正確，我們將迎來一場狂野的旅程。
Let me tell you what we see.
讓我告訴你我們看到的情況。
Table of Contents 目錄
Each essay is meant to stand on its own, though I’d strongly encourage reading the series as a whole. For a pdf version of the full essay series, click here.
每篇文章都應該獨立存在，但我強烈建議通讀整個系列。要獲取完整文章系列的 PDF 版本，請點擊這裡。
Introduction [this page] 介紹 [本頁]
History is live in San Francisco.
歷史在舊金山活生生地展現。
I. From GPT-4 to AGI: Counting the OOMs
I. 從 GPT-4 到 AGI：計算 OOMs
AGI by 2027 is strikingly plausible. GPT-2 to GPT-4 took us from ~preschooler to ~smart high-schooler abilities in 4 years. Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year), algorithmic efficiencies (~0.5 OOMs/year), and “unhobbling” gains (from chatbot to agent), we should expect another preschooler-to-high-schooler-sized qualitative jump by 2027.
到 2027 年實現通用人工智慧（AGI）是非常有可能的。從 GPT-2 到 GPT-4，我們在四年內從大約幼兒園水平提升到大約聰明的高中生水平。追蹤計算能力（每年約 0.5 個數量級）、算法效率（每年約 0.5 個數量級）和“解鎖”增益（從聊天機器人到代理），我們應該預期到 2027 年會有另一個從幼兒園到高中的質量飛躍。
II. From AGI to Superintelligence: the Intelligence Explosion
二、從人工通用智慧到超級智慧：智慧爆炸
AI progress won’t stop at human-level. Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into ≤1 year. We would rapidly go from human-level to vastly superhuman AI systems. The power—and the peril—of superintelligence would be dramatic.
人工智慧的進步不會止步於人類水平。數以億計的通用人工智慧（AGI）可以自動化人工智慧研究，將十年的算法進展（超過 5 個數量級）壓縮到一年以內。我們將迅速從人類水平進展到遠超人類的人工智慧系統。超級智能的力量和危險將是戲劇性的。
III. The Challenges 三、挑戰
IIIa. Racing to the Trillion-Dollar Cluster
IIIa. 競逐兆美元集群
The most extraordinary techno-capital acceleration has been set in motion. As AI revenue grows rapidly, many trillions of dollars will go into GPU, datacenter, and power buildout before the end of the decade. The industrial mobilization, including growing US electricity production by 10s of percent, will be intense.
最非凡的科技資本加速已經啟動。隨著人工智慧收入迅速增長，數萬億美元將在本世紀末前投入到 GPU、數據中心和電力建設中。包括美國電力生產增長數十個百分點在內的工業動員將會非常激烈。
IIIb. Lock Down the Labs: Security for AGI
IIIb. 鎖定實驗室：AGI 的安全性
The nation’s leading AI labs treat security as an afterthought. Currently, they’re basically handing the key secrets for AGI to the CCP on a silver platter. Securing the AGI secrets and weights against the state-actor threat will be an immense effort, and we’re not on track.
該國領先的人工智慧實驗室將安全問題置於次要地位。目前，他們基本上是在把通用人工智慧的關鍵秘密拱手讓給中共。要防止國家行為者威脅到通用人工智慧的秘密和權重，將是一項巨大的努力，而我們並未走在正軌上。
IIIc. Superalignment IIIc. 超級對齊
Reliably controlling AI systems much smarter than we are is an unsolved technical problem. And while it is a solvable problem, things could easily go off the rails during a rapid intelligence explosion. Managing this will be extremely tense; failure could easily be catastrophic.
可靠地控制比我們聰明得多的人工智慧系統是一個尚未解決的技術問題。儘管這是一個可以解決的問題，但在快速的智能爆炸期間，事情很容易失控。管理這一過程將極其緊張；失敗很可能是災難性的。
IIId. The Free World Must Prevail
自由世界必須勝利
Superintelligence will give a decisive economic and military advantage. China isn’t at all out of the game yet. In the race to AGI, the free world’s very survival will be at stake. Can we maintain our preeminence over the authoritarian powers? And will we manage to avoid self-destruction along the way?
超級智能將帶來決定性的經濟和軍事優勢。中國完全沒有出局。在通用人工智慧的競賽中，自由世界的生存將岌岌可危。我們能否保持對威權勢力的優勢？我們能否在此過程中避免自我毀滅？
IV. The Project 四、該項目
As the race to AGI intensifies, the national security state will get involved. The USG will wake from its slumber, and by 27/28 we’ll get some form of government AGI project. No startup can handle superintelligence. Somewhere in a SCIF, the endgame will be on.
隨著通用人工智慧競賽的加劇，國家安全機構將會介入。美國政府將從沉睡中醒來，到 2027/2028 年，我們將會看到某種形式的政府通用人工智慧項目。沒有任何初創公司能夠應對超級智能。在某個敏感隔離設施中，終局將會展開。
V. Parting Thoughts 五、臨別感言
What if we’re right? 如果我們是對的呢？
Next post in series: 下一篇文章：
I. From GPT-4 to AGI: Counting the OOMs
I. 從 GPT-4 到 AGI：計算 OOMs
While I used to work at OpenAI, all of this is based on publicly-available information, my own ideas, general field-knowledge, or SF-gossip.
雖然我曾在 OpenAI 工作，但這一切都是基於公開可用的信息、我自己的想法、一般領域知識或科幻小說的八卦。
Thank you to Collin Burns, Avital Balwit, Carl Shulman, Jan Leike, Ilya Sutskever, Holden Karnofsky, Sholto Douglas, James Bradbury, Dwarkesh Patel, and many others for formative discussions. Thank you to many friends for feedback on earlier drafts. Thank you to Joe Ronan for help with graphics, and Nick Whitaker for publishing help.
感謝 Collin Burns、Avital Balwit、Carl Shulman、Jan Leike、Ilya Sutskever、Holden Karnofsky、Sholto Douglas、James Bradbury、Dwarkesh Patel 和許多其他人進行的建設性討論。感謝許多朋友對早期草稿的反饋。感謝 Joe Ronan 在圖形方面的幫助，以及 Nick Whitaker 在出版方面的幫助。
Dedicated to Ilya Sutskever.
獻給伊利亞·蘇茨克弗。
