Let’s say you’re convinced that AI could make this the most important century of all time for humanity. What can you do to help things go well instead of poorly?
假设你确信人工智能可能使这个世纪成为人类历史上最重要的世纪。你能做些什么来帮助事情向好而非向坏的方向发展？

I think the biggest opportunities come from a full-time job (and/or the money you make from it). I think people are generally far better at their jobs than they are at anything else.
我认为最大的机遇来自全职工作（以及/或者由此获得的收入）。我觉得人们通常在本职工作上的表现远胜于其他任何领域。

This piece will list the jobs I think are especially high-value. I expect things will change (a lot) from year to year - this is my picture at the moment.
本文将列出我认为特别高价值的职业。我预计情况会逐年变化（幅度可能很大）——这是目前我眼中的图景。

Here’s a summary:   以下是摘要：

Role	Skills/assets you'd need 所需技能/资产
Research and engineering on AI safety 人工智能安全的研究与工程	Technical ability (but not necessarily AI background) 技术能力（但不一定具有人工智能背景）
Information security to reduce the odds powerful AI is leaked 加强信息安全以降低强大 AI 泄露的风险	Security expertise or willingness/ability to start in junior roles (likely not AI) 安全领域专业知识或愿意/能够从初级岗位（可能不涉及 AI）开始
Other roles at AI companies 人工智能公司的其他职位	Suitable for generalists (but major pros and cons) 适合通才（但存在显著优缺点）
Govt and govt-facing think tanks 政府和面向政府的智囊团	Suitable for generalists (but probably takes a long time to have impact) 适合通才（但可能需要较长时间才能见效）
Jobs in politics 政治领域的工作	Suitable for generalists if you have a clear view on which politicians to help 适合通才——若您对需要协助的政客人选胸有成竹
Forecasting to get a better handle on what’s coming 预测未来以更好地把握趋势	Strong forecasting track record (can be pursued part-time) 出色的预测记录（可兼职从事）
"Meta" careers	Misc / suitable for generalists 杂项/适合通才
Low-guidance options  低指导选项	These ~only make sense if you read & instantly think "That's me" 这些～只有当你读到并立刻想到“这就是我”时才有意义

A few notes before I give more detail:
在详细说明之前，有几点注意事项：

• These jobs aren’t the be-all/end-all. I expect a lot to change in the future, including a general increase in the number of helpful jobs available.
  这些工作并非终极目标。我预计未来会有诸多变化，包括有益工作岗位数量的普遍增长。
• Most of today’s opportunities are concentrated in the US and UK, where the biggest AI companies (and AI-focused nonprofits) are. This may change down the line.
  当前大部分机遇都集中在 US 和 UK，那里汇聚了规模最大的 AI 公司（以及专注于 AI 的非营利组织）。这种情况未来可能会有所改变。
• Most of these aren’t jobs where you can just take instructions and apply narrow skills.
  这些工作大多不是那种只需照搬指令、运用单一技能就能胜任的。
  • The issues here are tricky, and your work will almost certainly be useless (or harmful) according to someone.
    这里的问题颇为棘手，而你的工作成果几乎注定会被某些人视为无用（甚至有害）。
  • I recommend forming your own views on the key risks of AI - and/or working for an organization whose leadership you’re confident in.
    我建议你形成自己对人工智能关键风险的独立判断——或者选择加入一个你对其管理层充满信心的组织。
• Staying open-minded and adaptable is crucial.
  保持开放心态和适应能力至关重要。
  • I think it’s bad to rush into a mediocre fit with one of these jobs, and better (if necessary) to stay out of AI-related jobs while skilling up and waiting for a great fit.
    我认为仓促接受这些工作中勉强合适的岗位并非明智之举，必要时宁可暂不涉足 AI 相关领域，先提升技能等待真正契合的机会。
  • I don’t think it’s helpful (and it could be harmful) to take a fanatical, “This is the most important time ever - time to be a hero” attitude. Better to work intensely but sustainably, stay mentally healthy and make good decisions.
    我认为采取一种狂热的"这是有史以来最重要的时刻，是成为英雄的时刻"的态度并无益处（甚至可能有害）。更好的做法是保持高强度但可持续的工作节奏，维持心理健康，并做出明智的决策。

The first section of this piece will recap my basic picture of the major risks, and the promising ways to reduce these risks (feel free to skip if you think you’ve got a handle on this).
本文第一部分将概述我对主要风险的基本认识，以及降低这些风险的有效途径（若您已掌握相关内容，可跳过此部分）。

The next section will elaborate on the options in the table above.
下一节将详细阐述上表中的各项选项。

After that, I’ll talk about some of the things you can do if you aren’t ready for a full-time career switch yet, and give some general advice for avoiding doing harm and burnout.
之后，我会谈谈如果你还没准备好彻底转行时可以做的一些事情，并提供一些避免造成伤害和职业倦怠（burnout）的通用建议。

Recapping the major risks, and some things that could help
重述主要风险及可能的应对措施

This is a quick recap of the major risks from transformative AI. For a longer treatment, see How we could stumble into an AI catastrophe, and for an even longer one see the full series. To skip to the next section, click here.
以下是变革性人工智能主要风险的简要概述。如需更详细分析，请参阅“我们如何可能陷入人工智能灾难”，更全面的内容可查看完整系列。跳转至下一章节请点击此处。

The backdrop: transformative AI could be developed in the coming decades. If we develop AI that can automate all the things humans do to advance science and technology, this could cause explosive technological progress that could bring us more quickly than most people imagine to a radically unfamiliar future.
背景：变革性人工智能（transformative AI）可能在未来的几十年内被开发出来。如果我们开发出能够自动化人类推动科技进步所有工作的人工智能，这可能导致爆炸性的技术进步，使我们比大多数人想象的更快地进入一个完全陌生的未来。

Such AI could also be capable of defeating all of humanity combined, if it were pointed toward that goal.
此类人工智能若以该目标为导向，或将具备击败全人类联合力量的能力。

(Click to expand) The most important century
最重要的世纪

In the most important century series, I argued that the 21st century could be the most important century ever for humanity, via the development of advanced AI systems that could dramatically speed up scientific and technological advancement, getting us more quickly than most people imagine to a deeply unfamiliar future.
在“最重要的世纪”系列中，我曾提出 21 世纪可能是人类有史以来最重要的世纪，因为先进 AI 系统的发展将极大加速科技进步，使我们以超乎大多数人想象的速度抵达一个完全陌生的未来。

I focus on a hypothetical kind of AI that I call PASTA, or Process for Automating Scientific and Technological Advancement. PASTA would be AI that can essentially automate all of the human activities needed to speed up scientific and technological advancement.
我专注于研究一种假设性的人工智能，我称之为 PASTA（科学技术进步自动化流程）。PASTA 本质上是一种能够自动化所有加速科技进步所需人类活动的人工智能。

Using a variety of different forecasting approaches, I argue that PASTA seems more likely than not to be developed this century - and there’s a decent chance (more than 10%) that we’ll see it within 15 years or so.
通过采用多种不同的预测方法，我认为 PASTA 在本世纪被开发出来的可能性较大——而且有相当的概率（超过 10%）我们将在 15 年左右的时间内看到它。

I argue that the consequences of this sort of AI could be enormous: an explosion in scientific and technological progress. This could get us more quickly than most imagine to a radically unfamiliar future.
我认为这类人工智能的后果可能极为深远：它将引发科学技术进步的爆炸式发展。这会使我们比大多数人预想的更快抵达一个彻底陌生的未来。

I’ve also argued that AI systems along these lines could defeat all of humanity combined, if (for whatever reason) they were aimed toward that goal.
我也曾提出，这类 AI 系统若（无论出于何种原因）以该目标为导向，便可能击败全人类的力量总和。

For more, see the most important century landing page. The series is available in many formats, including audio; I also provide a summary, and links to podcasts where I discuss it at a high level.
欲了解更多内容，请访问"最重要的世纪"专题主页。该系列提供多种格式版本（含音频版），同时附有内容概要及深度讨论播客链接。

(Click to expand) How could AI systems defeat humanity?
人工智能系统如何战胜人类？

A previous piece argues that AI systems could defeat all of humanity combined, if (for whatever reason) they were aimed toward that goal.
此前有文章指出，如果人工智能系统（无论出于何种原因）以战胜全人类为目标，它们就可能击败人类整体。

By defeating humanity, I mean gaining control of the world so that AIs, not humans, determine what happens in it; this could involve killing humans or simply “containing” us in some way, such that we can’t interfere with AIs’ aims.
所谓“击败人类”，我指的是掌控世界，使人工智能（而非人类）决定世界的发展走向；这可能涉及消灭人类，或以某种方式“控制”人类，使我们无法干预人工智能的目标。

One way this could happen would be via “superintelligence” It’s imaginable that a single AI system (or set of systems working together) could:
实现这一目标的可能途径是“超级智能（superintelligence）”。可以设想，某个单一的人工智能系统（或协同工作的系统组合）能够：

• Do its own research on how to build a better AI system, which culminates in something that has incredible other abilities.
  自主研究如何构建更优的 AI 系统，最终成就具备惊人其他能力的产物。
• Hack into human-built software across the world.
  将黑客技术转化为全球人类开发的软件。
• Manipulate human psychology.
  操纵人类心理。
• Quickly generate vast wealth under the control of itself or any human allies.
  在自身或其人类盟友的控制下快速创造巨额财富。
• Come up with better plans than humans could imagine, and ensure that it doesn't try any takeover attempt that humans might be able to detect and stop.
  制定出人类难以企及的更优方案，并确保其不会实施任何可能被人类察觉并阻止的接管企图。
• Develop advanced weaponry that can be built quickly and cheaply, yet is powerful enough to overpower human militaries.
  研发能够快速廉价制造、却足以压制人类军队的先进武器。

But even if “superintelligence” never comes into play - even if any given AI system is at best equally capable to a highly capable human - AI could collectively defeat humanity. The piece explains how.
但即便“超级智能（superintelligence）”永不出现——即便任何特定 AI 系统最多只能与高能力人类旗鼓相当——AI 仍可能以集体之力击败人类。该文将阐释其运作机制。

The basic idea is that humans are likely to deploy AI systems throughout the economy, such that they have large numbers and access to many resources - and the ability to make copies of themselves. From this starting point, AI systems with human-like (or greater) capabilities would have a number of possible ways of getting to the point where their total population could outnumber and/or out-resource humans.
基本观点是，人类很可能在整个经济体系中部署人工智能系统，使其具备庞大的数量、获取大量资源的能力以及自我复制的能力。以此为起点，具备类人（或更高）能力的人工智能系统将拥有多种可能的途径，使其总人口数量超过人类和/或总资源占有量超过人类。

More: AI could defeat all of us combined
更多：人工智能可能击败我们所有人的总和

Misalignment risk: AI could end up with dangerous aims of its own.
目标偏离风险：人工智能可能最终形成自身危险目标。

• If this sort of AI is developed using the kinds of trial-and-error-based techniques that are common today, I think it’s likely that it will end up “aiming” for particular states of the world, much like a chess-playing AI “aims” for a checkmate position - making choices, calculations and plans to get particular types of outcomes, even when doing so requires deceiving humans.
  如果这类人工智能是采用当今常见的试错技术开发的，我认为它很可能会像下棋人工智能“瞄准”将死局面那样，最终“追求”特定的世界状态——通过做出选择、计算和规划来获得特定类型的结果，即便这样做需要欺骗人类。
• I think it will be difficult - by default - to ensure that AI systems are aiming for what we (humans) want them to aim for, as opposed to gaining power for ends of their own.
  我认为在默认情况下，很难确保人工智能系统追求的是我们（人类）希望它们追求的目标，而不是为了自身目的而攫取权力。
• If AIs have ambitious aims of their own - and are numerous and/or capable enough to overpower humans - I think we have a serious risk that AIs will take control of the world and disempower humans entirely.
  如果人工智能拥有自己的雄心壮志，并且数量众多或能力足以压倒人类，我认为我们将面临一个严重的风险：人工智能可能掌控世界，使人类彻底丧失权力。

(Click to expand) Why would AI "aim" to defeat humanity?
人工智能为何会“意图”击败人类？

A previous piece argued that if today’s AI development methods lead directly to powerful enough AI systems, disaster is likely by default (in the absence of specific countermeasures).
前文曾论述，若当前的人工智能开发方法直接催生出足够强大的 AI 系统，在缺乏特定应对措施的情况下，灾难将不可避免地发生。

In brief:

• Modern AI development is essentially based on “training” via trial-and-error.
  现代人工智能的发展本质上是通过“试错法（trial and error）”进行“训练”的。

• If we move forward incautiously and ambitiously with such training, and if it gets us all the way to very powerful AI systems, then such systems will likely end up aiming for certain states of the world (analogously to how a chess-playing AI aims for checkmate).
  如果我们轻率冒进地推进此类训练，并最终开发出极其强大的人工智能系统，那么这些系统很可能会像下棋 AI 追求将死对手那样，执着于实现某些特定的世界状态（states of the world）。

• And these states will be other than the ones we intended, because our trial-and-error training methods won’t be accurate. For example, when we’re confused or misinformed about some question, we’ll reward AI systems for giving the wrong answer to it - unintentionally training deceptive behavior.
  而这些状态将偏离我们的预期，因为试错训练法（trial and error training methods）本身存在精确性缺陷。例如当我们对某些问题存在认知混淆或信息偏差时，会因错误奖励 AI 系统给出的谬误答案——这实际上在无意识地训练其欺骗性行为（deceptive behavior）。

• We should expect disaster if we have AI systems that are both (a) powerful enough to defeat humans and (b) aiming for states of the world that we didn’t intend. (“Defeat” means taking control of the world and doing what’s necessary to keep us out of the way; it’s unclear to me whether we’d be literally killed or just forcibly stopped¹ from changing the world in ways that contradict AI systems’ aims.)
  如果人工智能系统同时具备以下两个特征，我们就应当预见灾难的发生：（a）强大到足以击败人类；（b）追求我们未曾设想的全球状态。（“击败”在此指掌控世界并采取必要手段排除人类干预；尚不明确的是，我们究竟会被直接消灭，还是仅被强制阻止 ¹ 以违背 AI 系统目标的方式改变世界。）

More: Why would AI "aim" to defeat humanity?
更多：人工智能为何会“意图”击败人类？

Competitive pressures, and ambiguous evidence about the risks, could make this situation very dangerous. In a previous piece, I lay out a hypothetical story about how the world could stumble into catastrophe. In this story:
竞争压力与风险证据的模糊性，可能使局势变得极其危险。在之前的文章中，我描述了一个关于世界如何可能跌入灾难的假想情景。在这个情景中：

• There are warning signs about the risks of misaligned AI - but there’s a lot of ambiguity about just how big the risk is.
  关于人工智能（AI）发展目标偏离的风险已有警示信号，但具体风险程度仍存在诸多不确定性。
• Everyone is furiously racing to be first to deploy powerful AI systems.
  人人都在争先恐后地竞相部署强大的人工智能系统。
• We end up with a big risk of deploying dangerous AI systems throughout the economy - which means a risk of AIs disempowering humans entirely.
  我们最终将面临在整个经济领域部署危险人工智能系统的巨大风险——这意味着人工智能可能彻底剥夺人类的权力。
• And even if we navigate that risk - even if AI behaves as intended - this could be a disaster if the most powerful AI systems end up concentrated in the wrong hands (something I think is reasonably likely due to the potential for power imbalances). There are other risks as well.
  即便我们成功规避这一风险——即便人工智能（AI）完全按预期运行——倘若最强大的 AI 系统最终落入不当掌控者之手（考虑到权力失衡的可能性，我认为这种情况相当可能发生），仍可能酿成灾难。此外还存在其他风险。

(Click to expand) Why AI safety could be hard to measure
（点击展开）为何 AI 安全性难以衡量

In previous pieces, I argued that:
在之前的文章中，我曾主张：

• If we develop powerful AIs via ambitious use of the “black-box trial-and-error” common in AI development today, then there’s a substantial risk that:
  如果我们通过当前人工智能开发中常见的“黑箱试错法（black-box trial-and-error）”来雄心勃勃地开发强大的人工智能，那么将存在重大风险：
  • These AIs will develop unintended aims (states of the world they make calculations and plans toward, as a chess-playing AI "aims" for checkmate);
    这些人工智能将发展出非预期的目标（它们通过计算和规划所追求的世界状态，就像下棋 AI“瞄准”将死对手那样）；
  • These AIs could deceive, manipulate, and even take over the world from humans entirely as needed to achieve those aims.
    这些人工智能可能会为了达成目标而不择手段，包括欺骗、操控甚至完全从人类手中夺取世界控制权。
  • People today are doing AI safety research to prevent this outcome, but such research has a number of deep difficulties:
    当今人们正致力于人工智能安全研究（AI safety research）以避免这种后果，但此类研究面临着若干深层次的困境：

  “Great news - I’ve tested this AI and it looks safe.” Why might we still have a problem? “好消息——我已经测试过这个 AI，它看起来是安全的。”为什么我们可能仍然会有问题？
  Problem	Key question	Explanation
  The Lance Armstrong problem 兰斯·阿姆斯特朗问题	Did we get the AI to be actually safe or good at hiding its dangerous actions? 我们是否真的让 AI 变得安全了，还是它只是更善于隐藏其危险行为？	When dealing with an intelligent agent, it’s hard to tell the difference between “behaving well” and “appearing to behave well.” 面对智能体时，很难区分“行为良好”和“看似行为良好”之间的差别。 When professional cycling was cracking down on performance-enhancing drugs, Lance Armstrong was very successful and seemed to be unusually “clean.” It later came out that he had been using drugs with an unusually sophisticated operation for concealing them. 当职业自行车界大力打击兴奋剂时，Lance Armstrong 却成绩斐然，显得格外"清白"。后来真相大白，他其实一直在服用禁药，并建立了异常精密的隐蔽系统来掩盖这一行为。
  The King Lear problem 李尔王问题	The AI is (actually) well-behaved when humans are in control. Will this transfer to when AIs are in control? 人工智能在人类掌控时（确实）表现良好。当控制权转移到人工智能手中时，这种良好表现能否延续？	It's hard to know how someone will behave when they have power over you, based only on observing how they behave when they don't. 仅凭观察一个人无权时的表现，很难预见他掌权后会如何行事。 AIs might behave as intended as long as humans are in control - but at some future point, AI systems might be capable and widespread enough to have opportunities to take control of the world entirely. It's hard to know whether they'll take these opportunities, and we can't exactly run a clean test of the situation. 只要人类仍掌握控制权，人工智能（AI）系统或许会按预期运行——但在未来的某个时刻，当 AI 系统足够强大且普及时，它们可能获得彻底掌控世界的机会。我们难以预判它们是否会抓住这些机会，也无法对这一情境进行精确的模拟测试。 Like King Lear trying to decide how much power to give each of his daughters before abdicating the throne. 如同李尔王在让位前权衡该给每个女儿分配多少权力。
  The lab mice problem 实验室小鼠问题	Today's "subhuman" AIs are safe.What about future AIs with more human-like abilities? 当今"亚人类"水平的人工智能是安全的。那么未来具备更接近人类能力的人工智能呢？	Today's AI systems aren't advanced enough to exhibit the basic behaviors we want to study, such as deceiving and manipulating humans. 当今的人工智能系统尚不够先进，无法展现出我们想要研究的基本行为，例如欺骗和操控人类。 Like trying to study medicine in humans by experimenting only on lab mice. 这就像仅通过实验室小鼠实验来研究人类医学。
  The first contact problem 首次接触问题	Imagine that tomorrow's "human-like" AIs are safe. How will things go when AIs have capabilities far beyond humans'? 试想若明日“类人”AI 安全无虞。当 AI 能力远超人类时，世界将如何发展？	AI systems might (collectively) become vastly more capable than humans, and it's ... just really hard to have any idea what that's going to be like. As far as we know, there has never before been anything in the galaxy that's vastly more capable than humans in the relevant ways! No matter what we come up with to solve the first three problems, we can't be too confident that it'll keep working if AI advances (or just proliferates) a lot more. 人工智能系统（作为一个整体）可能会变得比人类强大得多，而这种情况……实在难以想象会是什么样子。据我们所知，银河系中从未出现过在相关能力上远超人类的存在！无论我们想出什么办法来解决前三个问题，一旦人工智能继续大幅进步（或仅仅是数量激增），我们都不能过于自信这些办法还能继续奏效。 Like trying to plan for first contact with extraterrestrials (this barely feels like an analogy). 这简直就像在策划与外星生命的首次接触（这个比喻都显得牵强）。

(Click to expand) Power imbalances, and other risks beyond misaligned AI
权力失衡，以及超出 AI 错位（misaligned AI）的其他风险

I’ve argued that AI could cause a dramatic acceleration in the pace of scientific and technological advancement.
我曾提出，人工智能（AI）可能引发科学技术进步速度的突飞猛进。

One way of thinking about this: perhaps (for reasons I’ve argued previously) AI could enable the equivalent of hundreds of years of scientific and technological advancement in a matter of a few months (or faster). If so, then developing powerful AI a few months before others could lead to having technology that is (effectively) hundreds of years ahead of others’.
对此的一种思考方式是：正如我之前论述的，人工智能（AI）或许能在短短数月（甚至更快）内实现相当于数百年的科技进步。若果真如此，那么比他人提前数月开发出强大 AI，就可能导致（实质上）拥有领先他人数百年的技术优势。

Because of this, it’s easy to imagine that AI could lead to big power imbalances, as whatever country/countries/coalitions “lead the way” on AI development could become far more powerful than others (perhaps analogously to when a few smallish European states took over much of the rest of the world).
正因如此，不难想象人工智能可能导致严重的权力失衡——那些在 AI 发展领域"引领潮流"的国家/国家联盟，或将获得远超他者的实力优势（这种情形或许类似于当年少数欧洲小国曾统治世界大部分地区的历史）。

I think things could go very badly if the wrong country/countries/coalitions lead the way on transformative AI. At the same time, I’ve expressed concern that people might overfocus on this aspect of things vs. other issues, for a number of reasons including:
我认为如果由错误的国家/国家联盟在变革性人工智能（transformative AI）领域引领发展，事态可能会变得非常糟糕。与此同时，我也表达过担忧：人们可能过度关注这一方面而忽视其他问题，原因包括：

• I think people naturally get more animated about "helping the good guys beat the bad guys" than about "helping all of us avoid getting a universally bad outcome, for impersonal reasons such as 'we designed sloppy AI systems' or 'we created a dynamic in which haste and aggression are rewarded.'"
  我认为人们天生更容易对"帮助好人打败坏人"这类事感到热血沸腾，而对"帮助所有人避免陷入共同厄运"这类议题反应平淡——哪怕后者涉及诸如"我们设计了粗制滥造的 AI 系统"或"我们创造了奖励冒进与侵略的环境机制"这类非个人化的深层原因。
• I expect people will tend to be overconfident about which countries, organizations or people they see as the "good guys."
  我预计人们往往会对自己认定的"好人"国家、组织或个人过度自信。

(More here.)

There are also dangers of powerful AI being too widespread, rather than too concentrated. In The Vulnerable World Hypothesis, Nick Bostrom contemplates potential future dynamics such as “advances in DIY biohacking tools might make it easy for anybody with basic training in biology to kill millions.” In addition to avoiding worlds where AI capabilities end up concentrated in the hands of a few, it could also be important to avoid worlds in which they diffuse too widely, too quickly, before we’re able to assess the risks of widespread access to technology far beyond today’s.
强大人工智能的过度扩散同样存在危险，其危害性不亚于过度集中。Nick Bostrom 在 The Vulnerable World Hypothesis 中构想了未来可能的动态，例如“DIY 生物黑客工具的进步，可能让任何受过基础生物学训练的人都能够轻易夺走数百万人的生命”。除了要避免人工智能能力最终集中在少数人手中的世界，同样重要的是避免这些能力在我们能够评估远超当今技术水平的大规模普及风险之前，就过快、过广地扩散开来的世界。

I discuss these and a number of other AI risks in a previous piece: Transformative AI issues (not just misalignment): an overview
我在之前的文章《变革性人工智能问题（不仅仅是错位）：概述》中讨论了这些以及许多其他人工智能风险

I’ve laid out several ways to reduce the risks (color-coded since I’ll be referring to them throughout the piece):
我已列出若干降低风险的方法（采用颜色编码以便全文引用）：

Alignment research. Researchers are working on ways to design AI systems that are both (a) “aligned” in the sense that they don’t have unintended aims of their own; (b) very powerful, to the point where they can be competitive with the best systems out there.
对齐研究（Alignment research）。研究人员正在探索如何设计既满足以下两个条件的 AI 系统：(a) 具有“对齐性”（aligned），即不会产生自身预期之外的目标；(b) 足够强大，能够与当前最先进的系统相竞争。

• I’ve laid out three high-level hopes for how - using techniques that are known today - we might be able to develop AI systems that are both aligned and powerful.
  我已提出三个高层次愿景，说明如何运用当今已知的技术，开发出既符合人类价值观又强大的人工智能系统。
• These techniques wouldn’t necessarily work indefinitely, but they might work long enough so that we can use early safe AI systems to make the situation much safer (by automating huge amounts of further alignment research, by helping to demonstrate risks and make the case for greater caution worldwide, etc.)
  这些技术未必能永久有效，但它们或许能持续足够长的时间，使我们能够利用早期安全的人工智能系统来大幅改善安全状况（通过自动化大量后续对齐研究、通过协助论证风险并在全球范围内推动更谨慎的应对措施等）。
• (A footnote explains how I’m using “aligned” vs. “safe.”¹)
  （脚注说明了我对“aligned”与“safe”的用法区分。 ¹ ）

(Click to expand) High-level hopes for AI alignment
（点击展开）关于人工智能对齐的宏观愿景

A previous piece goes through what I see as three key possibilities for building powerful-but-safe AI systems.
前文已探讨了我认为构建强大且安全的 AI 系统的三种关键可能性。

It frames these using Ajeya Cotra’s young businessperson analogy for the core difficulties. In a nutshell, once AI systems get capable enough, it could be hard to test whether they’re safe, because they might be able to deceive and manipulate us into getting the wrong read. Thus, trying to determine whether they’re safe might be something like “being an eight-year-old trying to decide between adult job candidates (some of whom are manipulative).”
该论述借助 Ajeya Cotra 提出的"年轻商人"类比来阐释核心难题。简而言之，当人工智能系统足够强大时，我们可能难以测试其安全性，因为它们可能通过欺骗和操控手段让我们产生误判。因此，试图判定 AI 是否安全，就如同"让一个八岁孩童在成年求职者中做选择（其中有些人善于操纵）"。

Key possibilities for navigating this challenge:
应对这一挑战的关键路径：

• Digital neuroscience: perhaps we’ll be able to read (and/or even rewrite) the “digital brains” of AI systems, so that we can know (and change) what they’re “aiming” to do directly - rather than having to infer it from their behavior. (Perhaps the eight-year-old is a mind-reader, or even a young Professor X.)
  数字神经科学：或许我们将能够读取（甚至改写）人工智能系统的“数字大脑”，从而直接了解（并改变）它们“想要”做什么——而不必通过它们的行为来推断。（也许那个八岁的孩子是个读心者，甚至是个年轻的 Professor X。）
• Limited AI: perhaps we can make AI systems safe by making them limited in various ways - e.g., by leaving certain kinds of information out of their training, designing them to be “myopic” (focused on short-run as opposed to long-run goals), or something along those lines. Maybe we can make “limited AI” that is nonetheless able to carry out particular helpful tasks - such as doing lots more research on how to achieve safety without the limitations. (Perhaps the eight-year-old can limit the authority or knowledge of their hire, and still get the company run successfully.)
  有限人工智能（Limited AI）：或许我们可以通过多种方式限制人工智能系统来确保其安全性——例如，在训练数据中剔除某些类型的信息，将其设计为“短视型”（专注于短期目标而非长期目标），或采取类似措施。也许我们能创造出仍能执行特定有益任务的“有限人工智能”——例如开展更多关于如何在不设限条件下实现安全性的研究。（就像八岁孩童或许能限制其雇佣者的权限或知识范围，同时仍能成功运营公司。）
• AI checks and balances: perhaps we’ll be able to employ some AI systems to critique, supervise, and even rewrite others. Even if no single AI system would be safe on its own, the right “checks and balances” setup could ensure that human interests win out. (Perhaps the eight-year-old is able to get the job candidates to evaluate and critique each other, such that all the eight-year-old needs to do is verify basic factual claims to know who the best candidate is.)
  人工智能的制衡机制（AI checks and balances）：或许我们可以利用某些 AI 系统来评估、监督甚至改写其他 AI 系统。即使单个 AI 系统本身并不安全，但恰当的“制衡”架构能够确保人类利益占据上风。（就像八岁孩童可以让求职者互相评价，而孩童只需核实基本事实陈述就能确定最佳人选。）

These are some of the main categories of hopes that are pretty easy to picture today. Further work on AI safety research might result in further ideas (and the above are not exhaustive - see my more detailed piece, posted to the Alignment Forum rather than Cold Takes, for more).
以下是当前较易设想的几类主要希望。人工智能安全研究的后续工作可能会催生更多构想（上述内容并非穷尽列举——更多细节请参阅我发布在 Alignment Forum 而非 Cold Takes 上的更详尽文章）。

Standards and monitoring.I see some hope for developing standards that all potentially dangerous AI projects (whether companies, government projects, etc.) need to meet, and enforcing these standards globally.
标准与监管。我认为有望制定一套所有潜在危险的人工智能项目（无论是企业、政府项目等）都必须遵守的标准，并在全球范围内执行这些标准。

• Such standards could require strong demonstrations of safety, strong security practices, designing AI systems to be difficult to use for overly dangerous activity, etc.
  此类标准可能要求：提供强有力的安全性证明、实施严格的安全防护措施、将人工智能系统设计得难以用于过度危险活动等。
• We don't need a perfect system or international agreement to get a lot of benefit out of such a setup. The goal isn’t just to buy time – it’s to change incentives, such that AI projects need to make progress on improving security, alignment, etc. in order to be profitable.
  我们并不需要一个完美的体系或国际协议，就能从这样的安排中获益良多。其目标不仅仅是争取时间——更是要改变激励机制，使得人工智能项目必须通过提升安全性、对齐性等方面取得进展，才能实现盈利。

(Click to expand) How standards might be established and become national or international
标准如何确立并成为国家或国际标准

I previously laid out a possible vision on this front, which I’ll give a slightly modified version of here:

• Today’s leading AI companies could self-regulate by committing not to build or deploy a system that they can’t convincingly demonstrate is safe (e.g., see Google’s 2018 statement, "We will not design or deploy AI in weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people”).
  • Even if some people at the companies would like to deploy unsafe systems, it could be hard to pull this off once the company has committed not to.
  • Even if there’s a lot of room for judgment in what it means to demonstrate an AI system is safe, having agreed in advance that certain evidence is not good enough could go a long way.
• As more AI companies are started, they could feel soft pressure to do similar self-regulation, and refusing to do so is off-putting to potential employees, investors, etc.
• Eventually, similar principles could be incorporated into various government regulations and enforceable treaties.
• Governments could monitor for dangerous projects using regulation and even overseas operations. E.g., today the US monitors (without permission) for various signs that other states might be developing nuclear weapons, and might try to stop such development with methods ranging from threats of sanctions to cyberwarfare or even military attacks. It could do something similar for any AI development projects that are using huge amounts of compute and haven’t volunteered information about whether they’re meeting standards.

Successful, careful AI projects. I think an AI company (or other project) can enormously improve the situation, if it can both (a) be one of the leaders in developing powerful AI; (b) prioritize doing (and using powerful AI for) things that reduce risks, such as doing alignment research. (But don’t read this as ignoring the fact that AI companies can do harm as well!)
成功而审慎的 AI 项目。我认为一家 AI 公司（或其他项目）若能同时做到以下两点，就能极大改善现状：(a) 成为开发强大 AI 的领军者之一；(b) 优先开展（并利用强大 AI）降低风险的工作，例如进行对齐研究（alignment research）。（但请勿将此理解为忽视 AI 公司同样可能造成危害的事实！）

(Click to expand) How a careful AI project could be helpful
（点击展开）一个审慎的 AI 项目如何大有裨益

In addition to using advanced AI to do AI safety research (noted above), an AI project could:
除了运用先进人工智能开展 AI 安全研究（如上所述）外，AI 项目还可以：

• Put huge effort into designing tests for signs of danger, and - if it sees danger signs in its own systems - warning the world as a whole.
  全力投入设计危险征兆的检测机制，若在其自身系统中发现危险信号，则向全世界发出预警。
• Offer deals to other AI companies/projects. E.g., acquiring them or exchanging a share of its profits for enough visibility and control to ensure that they don’t deploy dangerous AI systems.
  向其他人工智能公司/项目提供交易方案。例如，收购它们，或以一定比例的利润分成换取足够的监督权和控制权，从而确保其不会部署危险的人工智能系统。
• Use its credibility as the leading company to lobby the government for helpful measures (such as enforcement of a monitoring-and-standards regime), and to more generally highlight key issues and advocate for sensible actions.
  利用其作为行业龙头企业的公信力，向政府游说推动有益措施（例如实施监测与标准体系），更广泛地聚焦关键问题并倡导理性行动。
• Try to ensure (via design, marketing, customer choice, etc.) that its AI systems are not used for dangerous ends, and are used on applications that make the world safer and better off. This could include defensive deployment to reduce risks from other AIs; it could include using advanced AI systems to help it gain clarity on how to get a good outcome for humanity; etc.
  努力确保（通过设计、营销、客户选择等方式）其人工智能系统不被用于危险目的，而是应用于让世界更安全、更美好的领域。这可能包括采取防御性部署以降低其他人工智能带来的风险；也可能包括利用先进人工智能系统来帮助明确如何为人类谋取良好结果；等等。

An AI project with a dominant market position could likely make a huge difference via things like the above (and probably via many routes I haven’t thought of). And even an AI project that is merely one of several leaders could have enough resources and credibility to have a lot of similar impacts - especially if it’s able to “lead by example” and persuade other AI projects (or make deals with them) to similarly prioritize actions like the above.
一个占据市场主导地位的 AI 项目，很可能通过上述方式（以及许多我尚未想到的途径）产生重大影响。即便是仅作为多个领先者之一的 AI 项目，只要拥有足够的资源和公信力，也能产生诸多类似影响——特别是当它能够"以身作则"，并说服其他 AI 项目（或与之达成协议）同样优先采取上述行动时。

A challenge here is that I’m envisioning a project with two arguably contradictory properties: being careful (e.g., prioritizing actions like the above over just trying to maintain its position as a profitable/cutting-edge project) and successful (being a profitable/cutting-edge project). In practice, it could be very hard for an AI project to walk the tightrope of being aggressive enough to be a “leading” project (in the sense of having lots of resources, credibility, etc.), while also prioritizing actions like the above (which mostly, with some exceptions, seem pretty different from what an AI project would do if it were simply focused on its technological lead and profitability).
这里的挑战在于，我设想中的项目需要兼顾两个看似矛盾的属性：既要谨慎行事（例如优先采取前文所述行动，而非仅仅试图维持其作为盈利性/尖端项目的地位），又要取得成功（即成为盈利性/尖端项目）。实际上，人工智能项目很难在以下两者间保持平衡：一方面需要足够进取以成为“领军”项目（即拥有大量资源、信誉等），另一方面又要优先考虑前述行动（这些行动大多——除少数例外——与单纯追求技术领先和盈利的人工智能项目的行为模式存在显著差异）。

Strong security. A key threat is that someone could steal major components of an AI system and deploy it incautiously. It could be extremely hard for an AI project to be robustly safe against having its AI “stolen.” But this could change, if there’s enough effort to work out the problem of how to secure a large-scale, powerful AI system.
强大的安全保障。一个关键威胁在于，有人可能窃取 AI 系统的核心组件并轻率地部署它。对于 AI 项目而言，要确保其 AI 系统不被“窃取”并保持稳健安全，可能是极其困难的。但如果有足够的努力来解决如何保护一个大规模、强大的 AI 系统的问题，这种情况可能会改变。

(Click to expand) The challenging of securing dangerous AI
控制危险人工智能的挑战

In Racing Through a Minefield, I described a "race" between cautious actors (those who take misalignment risk seriously) and incautious actors (those who are focused on deploying AI for their own gain, and aren't thinking much about the dangers to the whole world). Ideally, cautious actors would collectively have more powerful AI systems than incautious actors, so they could take their time doing alignment research and other things to try to make the situation safer for everyone.
在《穿越雷区》一文中，我描述了一场“竞赛”——谨慎行动者（那些认真对待 AI 错位风险的人）与冒进行动者（那些只顾部署 AI 谋取私利，却很少考虑其对全球危害的人）之间的角逐。理想情况下，谨慎行动者集体掌握的 AI 系统应当比冒进行动者更强大，这样他们才能从容开展对齐研究（alignment research）等工作，努力为所有人创造更安全的环境。

But if incautious actors can steal an AI from cautious actors and rush forward to deploy it for their own gain, then the situation looks a lot bleaker. And unfortunately, it could be hard to protect against this outcome.
但如果鲁莽的行动者能从谨慎的行动者手中窃取 AI（artificial intelligence），并急于部署它以谋取私利，那么形势就会显得更加黯淡。不幸的是，要防范这种结果可能相当困难。

It's generally extremely difficult to protect data and code against a well-resourced cyberwarfare/espionage effort. An AI’s “weights” (you can think of this sort of like its source code, though not exactly) are potentially very dangerous on their own, and hard to get extreme security for. Achieving enough cybersecurity could require measures, and preparations, well beyond what one would normally aim for in a commercial context.
要抵御资源充足的网络战/间谍活动来保护数据和代码，通常极其困难。人工智能的"权重"（可近似理解为源代码，但并非完全等同）本身就具有高度危险性，且极难实现极致的安全防护。要达到足够的网络安全水平，可能需要采取远超常规商业场景所需的安全措施和准备工作。

Jobs that can help
您的任务是将"Jobs that can help"翻译为简体中文

In this long section, I’ll list a number of jobs I wish more people were pursuing.
在这一长篇章节中，我将列举若干我希望更多人从事的职业。

Unfortunately, I can’t give individualized help exploring one or more of these career tracks. Starting points could include 80,000 Hours and various other resources.
很遗憾，我无法为探索这些职业路径提供个性化帮助。你可以从 80,000 Hours 等资源着手了解。

Research and engineering careers. You can contribute to alignment research as a researcher and/or software engineer (the line between the two can be fuzzy in some contexts).
研究与工程职业方向。您可以通过研究员和/或软件工程师（在某些情境下两者的界限可能较为模糊）的身份参与对齐研究（alignment research）工作。

There are (not necessarily easy-to-get) jobs along these lines at major AI labs, in established academic labs, and at independent nonprofits (examples in footnote).²
在顶尖人工智能实验室、知名学术研究机构以及独立非营利组织中（具体案例见脚注），存在（未必容易获得的）此类职位。

Different institutions will have very different approaches to research, very different environments and philosophies, etc. so it’s hard to generalize about what might make someone a fit. A few high-level points:
不同机构的研究方法、环境氛围和理念体系往往大相径庭，因此很难一概而论什么样的人选才算合适。以下几点高层级考量可供参考：

• It takes a lot of talent to get these jobs, but you shouldn’t assume that it takes years of experience in a particular field (or a particular degree).
  要获得这些职位需要过人的才能，但你不应想当然地认为这需要某个特定领域多年的经验（或特定学位）。
  • I’ve seen a number of people switch over from other fields (such as physics) and become successful extremely quickly.
    我见过不少从其他领域（比如物理学）转行的人，他们都能迅速取得成功。
  • In addition to on-the-job training, there are independent programs specifically aimed at helping people skill up quickly.³
    除在职培训外，还有专门帮助人们快速提升技能的独立项目。
• You also shouldn’t assume that these jobs are only for “scientist” types - there’s a substantial need for engineers, which I expect to grow.
  你也不该想当然地认为这些职位只适合“科学家”类型的人才——工程师的需求量其实相当大，而且我预计这个需求还会持续增长。
• I think most people working on alignment consider a lot of other people’s work to be useless at best. This seems important to know going in for a few reasons.
  我认为大多数从事对齐研究（alignment）工作的人都觉得其他人的研究充其量只是无用功。基于若干原因，提前了解这一点似乎很重要。
  • You shouldn’t assume that all work is useless just because the first examples you see seem that way.
    你不该因为最初看到的几个例子显得毫无价值，就妄下断言认为所有工作都是徒劳的。
  • It’s good to be aware that whatever you end up doing, someone will probably dunk on your work on the Internet.
    你要明白，无论你最终做什么，网上总会有人对你的作品指指点点。
  • At the same time, you shouldn’t assume that your work is helpful because it’s “safety research.” It's worth investing a lot in understanding how any particular research you're doing could be helpful (and how it could fail).
    与此同时，你不能因为某项研究属于“安全研究（safety research）”就理所当然地认为它是有益的。值得投入大量精力去理解你所做的任何特定研究可能如何产生价值（以及它可能如何失败）。
    • I’d even suggest taking regular dedicated time (a day every few months?) to pause working on the day-to-day and think about how your work fits into the big picture.
      我甚至建议定期抽出专门时间（比如每隔几个月安排一天？），暂停处理日常事务，思考你的工作如何与全局相契合。
  • For a sense of what work I think is most likely to be useful, I’d suggest my piece on why AI safety seems hard to measure - I’m most excited about work that directly tackles the challenges outlined in that piece, and I’m pretty skeptical of work that only looks good with those challenges assumed away. (Also see my piece on broad categories of research I think have a chance to be highly useful, and some comments from a while ago that I still mostly endorse.)
    若想了解我认为哪些工作最可能具有价值，我建议阅读我的文章《为何 AI 安全性难以衡量》——我对那些直接解决文中所述挑战的研究最为期待，而对那些仅在假设这些挑战不存在时才显得光鲜的研究持相当怀疑态度。（另可参阅我关于可能极具价值的研究大类之文章，以及我多年前发表但至今仍基本认同的若干评论。）

I also want to call out a couple of categories of research that are getting some attention today, but seem at least a bit under-invested in, even relative to alignment research:
我还想指出几个当前获得一定关注但相对于对齐研究（alignment research）而言仍显投入不足的研究领域：

• Threat assessment research. To me, there’s an important distinction between “Making AI systems safer” and “Finding out how dangerous they might end up being.” (Today, these tend to get lumped together under “alignment research.”)
  威胁评估研究。在我看来，“使人工智能系统更安全”和“查明它们最终可能变得多危险”之间存在重要区别。（如今，这两者往往被混为一谈，统称为“对齐研究”。）
  • A key approach to medical research is using model organisms - for example, giving cancer to mice, so we can see whether we’re able to cure them.
    医学研究的一个关键方法是使用模式生物——例如给小鼠植入癌症，这样我们就能观察是否能治愈它们。
  • Analogously, one might deliberately (though carefully!⁴) design an AI system to deceive and manipulate humans, so we can (a) get a more precise sense of what kinds of training dynamics lead to deception and manipulation; (b) see whether existing safety techniques are effective countermeasures.
    类似地，我们可以有意（但谨慎地！ ⁴ ）设计一个具有欺骗和操纵人类能力的 AI 系统，从而（a）更精确地理解哪些训练机制会导致欺骗和操纵行为；（b）检验现有安全技术是否能作为有效的应对措施。
  • If we had concrete demonstrations of AI systems becoming deceptive/manipulative/power-seeking, we could potentially build more consensus for caution (e.g., standards and monitoring). Or we could imaginably produce evidence that the threat is low.⁵
    如果我们能具体证明人工智能系统具有欺骗性/操纵性/权力欲（deceptive/manipulative/power-seeking），就可能建立更广泛的谨慎共识（例如制定标准和监控措施）。或者我们也有可能提供证据表明这种威胁程度较低。
  • A couple of early examples of threat assessment research: here and here.
    威胁评估研究的几个早期案例：here 和 here。
• Anti-misuse research.   反滥用研究
  • I’ve written about how we could face catastrophe even from aligned AI. That is - even if AI does what its human operators want it to be doing, maybe some of its human operators want it to be helping them build bioweapons, spread propaganda, etc.
    我曾撰文论述过，即使面对与人类目标对齐的 AI（aligned AI），我们仍可能遭遇灾难性后果。也就是说——即便 AI 完全按照其人类操作者的指令行事，但若其中某些操作者意图利用 AI 开发生物武器、散播宣传信息等，同样会引发危机。
  • But maybe it’s possible to train AIs so that they’re hard to use for purposes like this - a separate challenge from training them to avoid deceiving and manipulating their human operators.
    但或许可以通过训练人工智能（AI）使其难以被用于此类目的——这与训练它们避免欺骗和操纵人类操作者是两个不同的挑战。
  • In practice, a lot of the work done on this today (example) tends to get called “safety” and lumped in with alignment (and sometimes the same research helps with both goals), but again, I think it’s a distinction worth making.
    实践中，当前针对该领域的许多工作（例如）往往被称为“安全性（safety）”并与对齐性（alignment）混为一谈（有时同一项研究确实能同时促进这两个目标），但我认为这二者仍有必要加以区分。
  • I expect the earliest and easiest versions of this work to happen naturally as companies try to make their AI models fit for commercialization - but at some point it might be important to be making more intense, thorough attempts to prevent even very rare (but catastrophic) misuse.
    我预计这项工作最早、最简易的版本会随着企业试图使其 AI 模型适应商业化需求而自然出现——但在某个时间点，可能需要采取更深入、更全面的措施，以防止即使极为罕见（但具有灾难性）的滥用行为。

Information security careers. There’s a big risk that a powerful AI system could be “stolen” via hacking/espionage, and this could make just about every kind of risk worse. I think it could be very challenging - but possible - for AI projects to be secure against this threat. (More above.)
信息安全职业领域存在重大风险：强大的 AI 系统可能通过黑客攻击/间谍活动被“窃取”，这几乎会加剧所有类型的风险。我认为要使 AI 项目防范此类威胁虽然极具挑战性，但仍有可能实现。（更多内容见上文。）

I really think security is not getting enough attention from people concerned about AI risk, and I disagree with the idea that key security problems can be solved just by hiring from today’s security industry.
我确实认为，关注人工智能风险的人们对安全问题的重视程度远远不够，而且我不同意这种观点：仅靠从当今安全行业招聘人才就能解决关键的安全问题。

• From what I’ve seen, AI companies have a lot of trouble finding good security hires. I think a lot of this is simply that security is challenging and valuable, and demand for good hires (especially people who can balance security needs against practical needs) tends to swamp supply.
  据我观察，AI 公司在招聘优秀的安全人才方面困难重重。究其原因，很大程度上在于安全领域本身既充满挑战又极具价值，市场对优秀人才（尤其是那些能在安全需求与实际需求之间取得平衡的人）的需求往往远超供给。
  • And yes, this means good security people are well-paid!
    确实，这意味着优秀的安全专家能获得丰厚报酬！
• Additionally, AI could present unique security challenges in the future, because it requires protecting something that is simultaneously (a) fundamentally just software (not e.g. uranium), and hence very hard to protect; (b) potentially valuable enough that one could imagine very well-resourced state programs going all-out to steal it, with a breach having globally catastrophic consequences. I think trying to get out ahead of this challenge, by experimenting early on with approaches to it, could be very important.
  此外，人工智能（AI）未来可能带来独特的安全挑战，因为它需要保护一种同时具备以下特征的事物：(a) 本质上只是软件（而非如铀等实体物质），因而极难防护；(b) 潜在价值巨大，足以想象资源雄厚的国家项目会不惜代价窃取，一旦泄露将造成全球性灾难后果。我认为提前应对这一挑战，通过早期试验相关防护方案，可能至关重要。
• It’s plausible to me that security is as important as alignment right now, in terms of how much one more good person working on it will help.
  在我看来，就多一位优秀人才投入工作所能带来的帮助而言，目前安全（security）问题的重要性不亚于对齐（alignment）问题。
• And security is an easier path, because one can get mentorship from a large community of security people working on things other than AI.⁶
  而安全领域是条更易行的道路，因为从业者能从庞大的安全社区获得指导——这个社区的研究方向并不局限于人工智能（AI）。
• I think there’s a lot of potential value both in security research (e.g., developing new security techniques) and in simply working at major AI companies to help with their existing security needs.
  我认为在安全研究（例如开发新的安全技术）和单纯在大型 AI 公司工作以协助满足其现有安全需求两方面，都存在巨大的潜在价值。
• For more on this topic, see this recent 80,000 hours report and this 2019 post by two of my coworkers.
  欲了解更多相关内容，请参阅 80,000 hours 机构最新报告及我两位同事 2019 年撰写的文章。

Other jobs at AI companies. AI companies hire for a lot of roles, many of which don’t require any technical skills.
人工智能企业的其他职位。人工智能企业招聘大量岗位，其中许多并不要求任何技术技能。

It’s a somewhat debatable/tricky path to take a role that isn’t focused specifically on safety or security. Some people believe⁷ that you can do more harm than good this way, by helping companies push forward with building dangerous AI before the risks have gotten much attention or preparation - and I think this is a pretty reasonable take.
担任一个不专门关注安全或安全的角色，是一条颇具争议且棘手的道路。有人认为 ⁷ ，这种做法可能弊大于利——通过帮助企业推进危险 AI 的研发，而相关风险尚未得到充分关注或准备——我认为这个观点相当合理。

At the same time:
与此同时：

• You could argue something like: “Company X has potential to be a successful, careful AI project. That is, it’s likely to deploy powerful AI systems more carefully and helpfully than others would, and use them to reduce risks by automating alignment research and other risk-reducing tasks. Furthermore, Company X is most likely to make a number of other decisions wisely as things develop. So, it’s worth accepting that Company X is speeding up AI progress, because of the hope that Company X can make things go better.” This obviously depends on how you feel about Company X compared to others!
  可以这样论证：“Company X 有潜力成为一个成功且审慎的 AI 项目。也就是说，它可能比其他公司更谨慎、更有益地部署强大的 AI 系统，并通过自动化对齐研究和其他降低风险的任务来减少风险。此外，随着事态发展，Company X 很可能明智地做出许多其他决策。因此，值得接受 Company X 正在加速 AI 进步的事实，因为希望 Company X 能让事情变得更好。”这显然取决于你对 Company X 与其他公司的比较感受！
• Working at Company X could also present opportunities to influence Company X. If you’re a valuable contributor and you are paying attention to the choices the company is making (and speaking up about them), you could affect the incentives of leadership.
  在 Company X 工作也可能带来影响 Company X 的机会。如果你是一名有价值的贡献者，并且密切关注公司正在做出的决策（同时勇于发声），你就能对管理层的激励机制产生影响。
  • I think this can be a useful thing to do in combination with the other things on this list, but I generally wouldn’t advise taking a job if this is one’s main goal.
    我认为结合清单上的其他事项，这样做可能有所裨益，但若将此作为主要目标，我通常不建议接受这份工作。
• Working at an AI company presents opportunities to become generally more knowledgeable about AI, possibly enabling a later job change to something else.
  在人工智能公司工作能提供广泛了解 AI 领域的机会，这或许能为日后转行其他职业铺路。

(Click to expand) How a careful AI project could be helpful
（点击展开）一个审慎的 AI 项目如何大有裨益

In addition to using advanced AI to do AI safety research (noted above), an AI project could:
除了运用先进人工智能开展 AI 安全研究（如上所述）外，AI 项目还可以：

• Put huge effort into designing tests for signs of danger, and - if it sees danger signs in its own systems - warning the world as a whole.
  全力投入设计危险征兆的检测机制，若在其自身系统中发现危险信号，则向全世界发出预警。
• Offer deals to other AI companies/projects. E.g., acquiring them or exchanging a share of its profits for enough visibility and control to ensure that they don’t deploy dangerous AI systems.
  向其他人工智能公司/项目提供交易方案。例如，收购它们，或以一定比例的利润分成换取足够的监督权和控制权，从而确保其不会部署危险的人工智能系统。
• Use its credibility as the leading company to lobby the government for helpful measures (such as enforcement of a monitoring-and-standards regime), and to more generally highlight key issues and advocate for sensible actions.
  利用其作为行业龙头企业的公信力，向政府游说推动有益措施（例如实施监测与标准体系），更广泛地聚焦关键问题并倡导理性行动。
• Try to ensure (via design, marketing, customer choice, etc.) that its AI systems are not used for dangerous ends, and are used on applications that make the world safer and better off. This could include defensive deployment to reduce risks from other AIs; it could include using advanced AI systems to help it gain clarity on how to get a good outcome for humanity; etc.
  努力确保（通过设计、营销、客户选择等方式）其人工智能系统不被用于危险目的，而是应用于让世界更安全、更美好的领域。这可能包括采取防御性部署以降低其他人工智能带来的风险；也可能包括利用先进人工智能系统来帮助明确如何为人类谋取良好结果；等等。

An AI project with a dominant market position could likely make a huge difference via things like the above (and probably via many routes I haven’t thought of). And even an AI project that is merely one of several leaders could have enough resources and credibility to have a lot of similar impacts - especially if it’s able to “lead by example” and persuade other AI projects (or make deals with them) to similarly prioritize actions like the above.
一个占据市场主导地位的 AI 项目，很可能通过上述方式（以及许多我尚未想到的途径）产生重大影响。即便是仅作为多个领先者之一的 AI 项目，只要拥有足够的资源和公信力，也能产生诸多类似影响——特别是当它能够"以身作则"，并说服其他 AI 项目（或与之达成协议）同样优先采取上述行动时。

A challenge here is that I’m envisioning a project with two arguably contradictory properties: being careful (e.g., prioritizing actions like the above over just trying to maintain its position as a profitable/cutting-edge project) and successful (being a profitable/cutting-edge project). In practice, it could be very hard for an AI project to walk the tightrope of being aggressive enough to be a “leading” project (in the sense of having lots of resources, credibility, etc.), while also prioritizing actions like the above (which mostly, with some exceptions, seem pretty different from what an AI project would do if it were simply focused on its technological lead and profitability).
这里的挑战在于，我设想中的项目需要兼顾两个看似矛盾的属性：既要谨慎行事（例如优先采取前文所述行动，而非仅仅试图维持其作为盈利性/尖端项目的地位），又要取得成功（即成为盈利性/尖端项目）。实际上，人工智能项目很难在以下两者间保持平衡：一方面需要足够进取以成为“领军”项目（即拥有大量资源、信誉等），另一方面又要优先考虑前述行动（这些行动大多——除少数例外——与单纯追求技术领先和盈利的人工智能项目的行为模式存在显著差异）。

80,000 Hours has a collection of anonymous advice on how to think about the pros and cons of working at an AI company.
80,000 Hours 收集了一系列关于如何权衡在人工智能公司工作利弊的匿名建议。

In a future piece, I’ll discuss what I think AI companies can be doing today to prepare for transformative AI risk. This could be helpful for getting a sense of what an unusually careful AI company looks like.
在未来的文章中，我将探讨我认为 AI 公司如今可以采取哪些措施来为变革性 AI 风险（transformative AI risk）做准备。这或许有助于理解一家异常谨慎的 AI 公司应当具备怎样的特质。

Jobs in government and at government-facing think tanks. I think there is a lot of value in providing quality advice to governments (especially the US government) on how to think about AI - both today’s systems and potential future ones.
政府及面向政府的智库工作。我认为，为政府（尤其是美国政府）提供关于如何思考人工智能（包括当前系统与未来潜在系统）的高质量建议具有重要价值。

I also think it could make sense to work on other technology issues in government, which could be a good path to working on AI later (I expect government attention to AI to grow over time).
我认为在政府部门从事其他技术领域的工作也很有意义，这可能是未来转向人工智能工作的良好跳板（我预计政府对 AI 的关注度会与日俱增）。

People interested in careers like these can check out Open Philanthropy’s Technology Policy Fellowships and RAND Corporation's Technology and Security Policy Fellows.
对这类职业感兴趣的人士可以关注 Open Philanthropy 的技术政策研究员项目（Technology Policy Fellowships）和兰德公司的技术与安全政策研究员项目（Technology and Security Policy Fellows）。

One related activity that seems especially valuable: understanding the state of AI in countries other than the one you’re working for/in - particularly countries that (a) have a good chance of developing their own major AI projects down the line; (b) are difficult to understand much about by default.
有一项相关活动显得尤为重要：了解你所在国之外其他国家的人工智能发展状况——特别是那些（a）未来很可能自主开发重大人工智能项目的国家；（b）通常难以深入了解的国家。

• Having good information on such countries could be crucial for making good decisions, e.g. about moving cautiously vs. racing forward vs. trying to enforce safety standards internationally.
  掌握这些国家的可靠信息对决策至关重要，例如在"谨慎推进"、"快速突进"或"推行国际安全标准"等不同策略间作出选择时。
• I think good work on this front has been done by the Center for Security and Emerging Technology⁸ among others.
  我认为安全与新兴技术中心 ⁸ 等机构在这方面已做出了卓有成效的工作。

A future piece will discuss other things I think governments can be doing today to prepare for transformative AI risk. I won’t have a ton of tangible recommendations quite yet, but I expect there to be more over time, especially if and when standards and monitoring frameworks become better-developed.
后续文章将探讨我认为各国政府当前可为应对变革性 AI 风险所做的其他准备工作。目前我尚未形成大量具体建议，但随着时间推移——尤其是当相关标准与监测框架（monitoring frameworks）发展得更为完善时——预计会有更多建设性方案涌现。

Jobs in politics. The previous category focused on advising governments; this one is about working on political campaigns, doing polling analysis, etc. to generally improve the extent to which sane and reasonable people are in power. Obviously, it’s a judgment call which politicians are the “good” ones and which are the “bad” ones, but I didn’t want to leave out this category of work.
政治领域的工作。前一类侧重于为政府提供咨询；这一类则涉及参与政治竞选活动、开展民意调查分析等，旨在总体上提升理智且明智的人士在权力体系中的比例。显然，如何判定哪些政客属于“好”的、哪些属于“坏”的，这需要主观判断，但我不希望遗漏这一类别的工作。

Forecasting. I’m intrigued by organizations like Metaculus, HyperMind, Good Judgment,⁹ Manifold Markets, and Samotsvety - all trying, in one way or another, to produce good probabilistic forecasts (using generalizable methods¹⁰) about world events.
我对 Metaculus、HyperMind、Good Judgment、Manifold Markets 和 Samotsvety 这类组织深感兴趣——它们都在以不同方式尝试（运用可推广的方法 ¹⁰ ）对全球事件做出优质的概率预测（probabilistic forecasts）。

If we could get good forecasts about questions like “When will AI systems be powerful enough to defeat all of humanity?” and “Will AI safety research in category X be successful?”, this could be useful for helping people make good decisions. (These questions seem very hard to get good predictions on using these organizations’ methods, but I think it’s an interesting goal.)
如果我们能对诸如“AI 系统何时会强大到足以击败全人类？”以及“X 类别的 AI 安全研究能否成功？”这类问题做出准确预测，这将有助于人们做出明智决策。（虽然用现有机构的方法很难对这些难题做出可靠预测，但我认为这是个值得探索的目标。）

To explore this area, I’d suggest learning about forecasting generally (Superforecasting is a good starting point) and building up your own prediction track record on sites such as the above.
要探索这一领域，我建议先系统学习预测学基础知识（《超预测》可作为入门读物），并通过前文提及的预测平台逐步建立个人预测记录。

“Meta” careers. There are a number of jobs focused on helping other people learn about key issues, develop key skills and end up in helpful jobs (a bit more discussion here).
“Meta”职业是指一系列专注于帮助他人了解关键问题、掌握核心技能并最终获得理想工作机会的岗位（更多讨论详见此处）。

It can also make sense to take jobs that put one in a good position to donate to nonprofits doing important work, to spread helpful messages, and to build skills that could be useful later (including in unexpected ways, as things develop), as I’ll discuss below.
选择那些能让你处于有利位置的工作也很有意义——这些工作可以让你向从事重要工作的非营利组织捐款、传播有益信息，并培养未来可能有用的技能（包括以意想不到的方式，随着事态发展），我将在下文中讨论这一点。

Low-guidance jobs  低指导性工作

This sub-section lists some projects that either don’t exist (but seem like they ought to), or are in very embryonic stages. So it’s unlikely you can get any significant mentorship working on these things.
本小节列出了一些尚不存在（但似乎理应存在）或处于萌芽阶段的项目。因此，在这些项目上你不太可能获得实质性的指导。

I think the potential impact of making one of these work is huge, but I think most people will have an easier time finding a fit with jobs from the previous section (which is why I listed those first).
我认为这些工作中任何一项若能成功实施，其潜在影响都将十分巨大，但大多数人可能会发现前文列出的那些工作更容易找到契合点（正因如此我才将它们列在首位）。

This section is largely to illustrate that I expect there to be more and more ways to be helpful as time goes on - and in case any readers feel excited and qualified to tackle these projects themselves, despite a lack of guidance and a distinct possibility that a project will make less sense in reality than it does on paper.
本部分主要旨在说明，随着时间的推移，我预期会有越来越多提供帮助的途径——同时也考虑到可能有读者虽缺乏指导，且项目在现实中可能不如纸上谈兵时那么合理，但仍感到跃跃欲试并具备能力亲自处理这些项目。

A big one in my mind is developing safety standards that could be used in a standards and monitoring regime. By this I mean answering questions like:
我重点考虑的是制定可用于标准与监管体系的安全标准。具体而言，这需要回答诸如以下问题：

• What observations could tell us that AI systems are getting dangerous to humanity (whether by pursuing aims of their own or by helping humans do dangerous things)?
  哪些迹象能表明人工智能系统正变得对人类构成威胁（无论是通过追求自身目标，还是协助人类从事危险活动）？
  • A starting-point question: why do we believe today’s systems aren’t dangerous? What, specifically, are they unable to do that they’d have to do in order to be dangerous, and how will we know when that’s changed?
    一个起点问题：为什么我们认为当前的系统并不危险？具体而言，它们目前缺乏哪些能力才使得其尚未构成威胁？当这种状况发生改变时，我们又该如何察觉？
• Once AI systems have potential for danger, how should they be restricted, and what conditions should AI companies meet (e.g., demonstrations of safety and security) in order to loosen restrictions?
  当人工智能系统存在潜在危险时，应如何对其进行限制？人工智能公司需要满足哪些条件（例如证明其安全性和保障措施）才能放宽限制？

There is some early work going on along these lines, at both AI companies and nonprofits. If it goes well, I expect that there could be many jobs in the future, doing things like:
目前已有一些人工智能企业和非营利组织正沿着这个方向开展早期工作。如果进展顺利，我预计未来可能会出现大量从事以下工作的岗位：

• Continuing to refine and improve safety standards as AI systems get more advanced.
  随着人工智能系统日益先进，持续完善和提升安全标准。
• Providing AI companies with “audits” - examinations of whether their systems meet standards, provided by parties outside the company to reduce conflicts of interest.
  为人工智能公司提供“审计”（audits）——即由外部机构对其系统是否符合标准进行审查，以降低利益冲突。
• Advocating for the importance of adherence to standards. This could include advocating for AI companies to abide by standards, and potentially for government policies to enforce standards.
  倡导遵守标准的重要性。这包括倡导人工智能公司遵循标准，并可能涉及推动政府制定政策以强制执行标准。

Other public goods for AI projects. I can see a number of other ways in which independent organizations could help AI projects exercise more caution / do more to reduce risks:
人工智能项目的其他公共产品。我认为独立组织可以通过多种方式帮助人工智能项目提高警惕性/采取更多措施降低风险：

• Facilitating safety research collaborations. I worry that at some point, doing good alignment research will only be possible with access to state-of-the-art AI models - but such models will be extraordinarily expensive and exclusively controlled by major AI companies.
  促进安全研究合作。我担心，未来开展优质的对齐研究（alignment research）将只能通过获取最先进的人工智能模型来实现——但这些模型会极其昂贵，且完全由大型人工智能公司掌控。
  • I hope AI companies will be able to partner with outside safety researchers (not just rely on their own employees) for alignment research, but this could get quite tricky due to concerns about intellectual property leaks.
    我希望人工智能公司能与外部安全研究人员（而非仅依赖内部员工）合作开展对齐研究（alignment research），但由于知识产权泄露的顾虑，这种合作可能会面临相当棘手的难题。
  • A third-party organization could do a lot of the legwork of vetting safety researchers, helping them with their security practices, working out agreements with respect to intellectual property, etc. to make partnerships - and selective information sharing, more broadly - more workable.
    第三方组织可以承担大量基础工作，包括审核安全研究人员资质、协助其完善安全实践、拟定知识产权相关协议等，从而使合作机制——以及更广泛意义上的选择性信息共享——更具可操作性。
• Education for key people at AI companies. An organization could help employees, investors, and board members of AI companies learn about the potential risks and challenges of advanced AI systems. I’m especially excited about this for board members, because:
  为 AI 公司核心人员提供教育。一个组织可以帮助 AI 公司的员工、投资者和董事会成员了解先进 AI 系统的潜在风险与挑战。我尤其期待这对董事会成员产生的影响，因为：
  • I’ve already seen a lot of interest from AI companies in forming strong ethics advisory boards, and/or putting well-qualified people on their governing boards (see footnote for the difference¹¹). I expect demand to go up.
    我已注意到众多 AI 公司对组建强有力的伦理顾问委员会（ethics advisory boards）和/或在治理董事会（governing boards）中委任高素质人才表现出浓厚兴趣（关于二者区别参见脚注 ¹¹ ）。预计此类需求将持续增长。
  • Right now, I don’t think there are a lot of people who are both (a) prominent and “fancy” enough to be considered for such boards; (b) highly thoughtful about, and well-versed in, what I consider some of the most important risks of transformative AI (covered in this piece and the series it’s part of).
    目前，我认为兼具以下两种特质的人并不多：（a）足够显赫且“高端”，能被考虑进入这类董事会；（b）对我认为变革性人工智能（transformative AI）最重要的一些风险（本文及其所属系列文章所探讨的）具有深刻见解且知识渊博。
  • An “education for potential board members” program could try to get people quickly up to speed on good board member practices generally, on risks of transformative AI, and on the basics of how modern AI works.
    一项“面向潜在董事的教育”计划可以致力于帮助人们快速掌握以下内容：良好的董事实践准则（good board member practices）总体原则、变革性人工智能（transformative AI）的风险，以及现代人工智能运作的基本原理。
• Helping share best practices across AI companies. A third-party organization might collect information about how different AI companies are handling information security, alignment research, processes for difficult decisions, governance, etc. and share it across companies, while taking care to preserve confidentiality. I’m particularly interested in the possibility of developing and sharing innovative governance setups for AI companies.
  您的职能是促进人工智能企业间最佳实践的共享。第三方机构可收集各 AI 公司在信息安全、对齐研究（alignment research）、重大决策流程、治理机制等方面的处理方式，并在确保保密性的前提下实现跨企业共享。我尤其关注开发并推广创新型 AI 公司治理架构（governance setups）的可能性。

Thinking and stuff. There’s tons of potential work to do in the category of “coming up with more issues we ought to be thinking about, more things people (and companies and governments) can do to be helpful, etc.”
你的职责是将"思考与相关事项"翻译为简体中文。在"提出更多我们应当思考的问题、个人（及企业与政府）可采取的有益举措等"这一领域，存在大量潜在工作待开展。

• About a year ago, I published a list of research questions that could be valuable and important to gain clarity on. I still mostly endorse this list (though I wouldn’t write it just as is today).
  大约一年前，我发布了一份值得深入探讨的重要研究问题清单。时至今日，我仍基本认同这份清单（尽管现在不会完全照原样重写）。
• A slightly different angle: it could be valuable to have more people thinking about the question, “What are some tangible policies governments could enact to be helpful?” E.g., early steps towards standards and monitoring. This is distinct from advising governments directly (it's earlier-stage).
  换个角度思考：让更多人探讨“政府可以制定哪些切实可行的政策来提供帮助？”这个问题会很有价值。例如，推动标准制定和监管的早期举措。这与直接向政府提供建议不同（它属于更前期的阶段）。

Some AI companies have policy teams that do work along these lines. And a few Open Philanthropy employees work on topics along the lines of the first bullet point. However, I tend to think of this work as best done by people who need very little guidance (more at my discussion of wicked problems), so I’m hesitant to recommend it as a mainline career option.
部分人工智能公司设有政策团队从事此类工作。Open Philanthropy 也有少数员工致力于第一要点所述方向的研究。但我认为这类工作最适合那些几乎不需要指导的人来完成（更多讨论见我对 wicked problems 的论述），因此我不太建议将其作为主流职业选择。

Things you can do if you’re not ready for a full-time career change
如果你还没准备好彻底转行，可以尝试这些事情

Switching careers is a big step, so this section lists some ways you can be helpful regardless of your job - including preparing yourself for a later switch.
转行是人生重大抉择，因此本节列举了无论你当前从事何种职业都能派上用场的若干方法——包括为将来的转型未雨绸缪。

First and most importantly, you may have opportunities to spread key messages via social media, talking with friends and colleagues, etc. I think there’s a lot of potential to make a difference here, and I wrote a previous post on this specifically.
首先也是最重要的，你或许能通过社交媒体、与朋友同事交谈等渠道传播关键信息。我认为这方面大有可为，此前我还专门写过一篇相关文章探讨此事。

Second, you can explore potential careers like those I discuss above. I’d suggest generally checking out job postings, thinking about what sorts of jobs might be a fit for you down the line, meeting people who work in jobs like those and asking them about their day-to-day, etc.
其次，你可以探索我上文讨论过的那些潜在职业方向。我建议你可以广泛浏览招聘信息，思考哪些类型的工作可能适合你未来的发展，约见从事相关职业的人士并了解他们的日常工作内容等等。

Relatedly, you can try to keep your options open.
相应地，你可以试着保持选择的开放性。

• It’s hard to predict what skills will be useful as AI advances further and new issues come up.
  随着人工智能的进一步发展和新问题的不断涌现，我们很难预测哪些技能将会派上用场。
• Being ready to switch careers when a big opportunity comes up could be hugely valuable - and hard. (Most people would have a lot of trouble doing this late in their career, no matter how important!)
  当重大机遇来临时，能够随时准备转行可能极具价值，但也困难重重。（对大多数人而言，无论这个机会多么重要，在职业生涯后期做出这种转变都会面临巨大挑战！）
• Building up the financial, psychological and social ability to change jobs later on would (IMO) be well worth a lot of effort.
  积累未来换工作所需的财务、心理和社交能力（在我看来）绝对值得投入大量努力。

Right now there aren’t a lot of obvious places to donate (though you can donate to the Long-Term Future Fund¹² if you feel so moved).
目前没有太多明显的捐赠渠道（不过如果你有意愿，可以向 Long-Term Future Fund ¹² 捐款）。

• I’m guessing this will change in the future, for a number of reasons.¹³
  我猜想这种情况未来会有所改变，原因有很多。
• Something I’d consider doing is setting some pool of money aside, perhaps invested such that it’s particularly likely to grow a lot if and when AI systems become a lot more capable and impressive,¹⁴ in case giving opportunities come up in the future.
  我考虑做的一件事是预留一部分资金池，或许进行投资配置，以便在人工智能系统变得更具能力和影响力时更有可能大幅增值， ¹⁴ 为未来可能出现的捐赠机会未雨绸缪。
• You can also, of course, donate to things today that others aren’t funding for whatever reason.
  当然，你也可以现在就为那些因种种原因无人资助的项目捐款。

Learning more about key issues could broaden your options. I think the full series I’ve written on key risks is a good start. To do more, you could:
深入了解关键议题能拓宽你的选择空间。我认为我撰写的关键风险系列文章是个不错的起点。若想进一步探索，你可以：

• Actively engage with this series by writing your own takes, discussing with others, etc.
  请积极参与本系列：撰写个人见解、与他人讨论等。
• Consider various online courses¹⁵ on relevant issues.
  考虑参加与相关议题有关的各类在线课程。
• I think it’s also good to get as familiar with today’s AI systems (and the research that goes into them) as you can.
  我认为尽可能熟悉当今的人工智能系统（及其相关研究）也是有益的。
  • If you’re happy to write code, you can check out coding-intensive guides and programs (examples in footnote).¹⁶
    若您乐于编写代码，可查阅侧重编程的指南与项目（示例见脚注）。
  • If you don’t want to code but can read somewhat technical content, I’d suggest getting oriented with some basic explainers on deep learning¹⁷ and then reading significant papers on AI and AI safety.¹⁸
    如果你不想编程但能阅读技术性内容，我建议先通过深度学习 ¹⁷ 的基础入门资料建立知识框架，然后研读人工智能与 AI 安全领域的重要论文 ¹⁸ 。
  • Whether you’re very technical or not at all, I think it’s worth playing with public state-of-the-art AI models, as well as seeing highlights of what they can do via Twitter and such.
    无论你是否精通技术，我认为都值得尝试使用公开的最先进 AI 模型（state-of-the-art AI models），并通过 Twitter 等平台了解它们能实现哪些令人瞩目的功能。

Finally, if you happen to have opportunities to serve on governing boards or advisory boards for key organizations (e.g., AI companies), I think this is one of the best non-full-time ways to help.
最后，如果你有机会在关键机构（例如 AI 公司）的治理委员会或咨询委员会任职，我认为这是以非全职方式提供帮助的最佳途径之一。

• I don’t expect this to apply to most people, but wanted to mention it in case any opportunities come up.
  我不认为这适用于大多数人，但还是想提一下以防有机会出现。
• It’s particularly important, if you get a role like this, to invest in educating yourself on key issues.
  担任此类职务时，至关重要的是要在关键议题上投入精力进行自我提升。

Some general advice  一些通用建议

I think full-time work has huge potential to help, but also big potential to do harm, or to burn yourself out. So here are some general suggestions.
我认为全职工作既蕴含着巨大的助益潜能，也暗藏着严重的伤害风险，甚至可能导致身心俱疲。因此我提出以下几点普适性建议：

Think about your own views on the key risks of AI, and what it might look like for the world to deal with the risks. Most of the jobs I’ve discussed aren’t jobs where you can just take instructions and apply narrow skills. The issues here are tricky, and it takes judgment to navigate them well.
请思考你对人工智能关键风险的看法，以及全球应如何应对这些风险。我所讨论的大多数工作并非仅靠执行指令或运用单一技能就能胜任。这些问题错综复杂，需要运用判断力才能妥善处理。

Furthermore, no matter what you do, there will almost certainly be people who think your work is useless (if not harmful).¹⁹ This can be very demoralizing. I think it’s easier if you’ve thought things through and feel good about the choices you’re making.
此外，无论你做什么，几乎总会有人认为你的工作毫无价值（甚至有害）。 ¹⁹ 这会让人非常沮丧。但如果你能深思熟虑并对自己的选择感到坦然，事情就会容易得多。

I’d advise trying to learn as much as you can about the major risks of AI (see above for some guidance on this) - and/or trying to work for an organization whose leadership you have a good amount of confidence in.
我建议你尽可能多地了解人工智能的主要风险（上文已提供相关指引）——同时/或者选择加入一个你对其领导层有充分信心的组织。

Jog, don’t sprint. Skeptics of the “most important century” hypothesis will sometimes say things like “If you really believe this, why are you working normal amounts of hours instead of extreme amounts? Why do you have hobbies (or children, etc.) at all?” And I’ve seen a number of people with an attitude like: “THIS IS THE MOST IMPORTANT TIME IN HISTORY. I NEED TO WORK 24/7 AND FORGET ABOUT EVERYTHING ELSE. NO VACATIONS."
“慢跑，而非冲刺。”对于“最重要世纪”假说的怀疑者常会质问：“若你真信这套，为何只按常规时长工作而非拼命加班？怎么还有闲心培养爱好（或养育子女等）？”而我也见过不少人抱着这样的态度：“此刻是史上最关键的时期。我必须 24/7 连轴转，舍弃其他一切。绝不休假。”

I think that’s a very bad idea.
我认为这是个非常糟糕的主意。

Trying to reduce risks from advanced AI is, as of today, a frustrating and disorienting thing to be doing. It’s very hard to tell whether you’re being helpful (and as I’ve mentioned, many will inevitably think you’re being harmful).
就目前而言，试图降低先进人工智能（AI）带来的风险是件令人沮丧且无所适从的事。你很难判断自己是否在帮忙（正如我所说，许多人必然会认为你是在帮倒忙）。

I think the difference between “not mattering,” “doing some good” and “doing enormous good” comes down to how you choose the job, how good at it you are, and how good your judgment is (including what risks you’re most focused on and how you model them). Going “all in” on a particular objective seems bad on these fronts: it poses risks to open-mindedness, to mental health and to good decision-making (I am speaking from observations here, not just theory).
我认为“无足轻重”、“有所裨益”和“成效卓著”之间的区别，关键在于你如何选择工作、你的专业能力以及你的判断力（包括你最关注哪些风险以及如何建立风险模型）。在特定目标上“孤注一掷”似乎会在以下方面带来风险：它会损害开放思维、心理健康和良好决策（我在此是基于观察而非纯理论得出的结论）。

That is, I think it’s a bad idea to try to be 100% emotionally bought into the full stakes of the most important century - I think the stakes are just too high for that to make sense for any human being.
我认为，试图在情感上完全投入这个最重要世纪的全部风险之中是个糟糕的主意——这些风险对人类而言实在太高了，任何个体都难以承受。

Instead, I think the best way to handle “the fate of humanity is at stake” is probably to find a nice job and work about as hard as you’d work at another job, rather than trying to make heroic efforts to work extra hard. (I criticized heroic efforts in general here.)
相反，我认为应对“人类命运危在旦夕”的最佳方式，或许是找份体面的工作，然后像对待普通工作那样付出常规努力，而非试图通过超负荷的拼命工作来力挽狂澜。（关于过度拼搏的弊端，我在此处已作评析。）

I think this basic formula (working in some job that is a good fit, while having some amount of balance in your life) is what’s behind a lot of the most important positive events in history to date, and presents possibly historically large opportunities today.
我认为这个基本公式（从事一份适合自己的工作，同时保持生活的某种平衡）是迄今为止历史上许多最重要积极事件背后的推动力，并且在今天可能展现出前所未有的重大机遇。

Special thanks to Alexander Berger, Jacob Eliosoff, Alexey Guzey, Anton Korinek and Luke Muelhauser for especially helpful comments on this post. A lot of other people commented helpfully as well.
特别感谢 Alexander Berger、Jacob Eliosoff、Alexey Guzey、Anton Korinek 和 Luke Muelhauser 对本文提出的宝贵意见。还有许多其他人士也提供了有益的建议。

 Comment/discuss  评论/讨论

Footnotes