38 min 2024-09
AI, policy, and the weird sci-fi future with Anthropic's Jack Clark
概要
Anthropic联合创始人Jack Clark谈AI的流氓国家理论、机器时间vs人类时间的风险论证、以及为什么今天的加速主义者其实是技术悲观者
核心洞察
- AI 系统不是工具而是"silicon countries"——它们拥有人工直觉、价值观和创造力，应以"全政府响应"（whole-of-government response）对待，而非分散到不同技术监管部门
- 机器时间 vs 人类时间是最有说服力的 AI 风险论证——人类思考速度仅 10 bits/sec（Caltech 论文），军事中 OODA loop 快者胜，对比机器速度差距比国家间军事差距更大
- "今天的加速主义者实际上是技术悲观者"——他们假设技术只比现在再前进一小步就停止，真正的加速主义者应对未来感到"shock, awe and some small amount of dread"
- 即使 AI 发展今天就停止，经济价值仍然巨大——类比"刚发现电力、装了第一批灯泡"，还有无数工厂待建、无数青少年会找到我们想不到的用途
- 2025年将是政策爆发之年：法国峰会、EU AI Act 实施、多国安全机构建立、美国新政府100天内将对 AI 采取行动
Jack Clark 的路径：从技术记者到"AI 唯一记者"到政策负责人
核心论点：通过深入理解技术来报道技术（"method acting journalism"），让 Jack 在2010年就看到 Google 将在数据中心上部署 ML
- 做记者时像"方法派演员"：写数据库就学 SQL，写芯片就学半导体制造
- "The Clark Side of the Cloud"系列：在全球巡游数据中心
- 2010年关键洞察：Google 建了这么多计算机，一定会用 ML 做些什么
- 2012年搬到美国，自封"世界唯一神经网络记者"——当时确实容易做到
- 2016年加入 OpenAI，迅速意识到政策的重要性——"how little people in policy knew about it"
- Import AI 新闻通讯：每周末写，附 Tech Tales 短篇小说，近10年约350篇
英国工党与 AI：继承安全机构而非推翻、聚焦经济增长
核心论点：UK 工党罕见地继承了前任保守党的 AI Safety Institute（AISI），同时将 AI 定位为经济增长引擎——关注"life and limb"类风险但更急于整合 AI 到政府运作中
- 工党刚执政两个月，将 AI 列为推动英国经济增长的关键技术
- 保守党建立的 AISI 被视为"national asset"——工党在思考如何扩展而非废除
- Anthropic 已与 AISI 做过部署前测试
- 工党 MP 关注的风险措辞："life and limb"——类似美国聚焦灾难性风险（生物/网络）
- Jack 在工党大会的策略："没选择打开恐怖手提箱"，而是谈 AISI 如何继续繁荣
- 政府对 AI 最兴奋的是后台应用：NHS 文书流转、议员选区回复处理、官僚系统优化
AI 的经济潜力：即使今天停止进步，价值仍然巨大
核心论点：当前 AI 如同"刚发现电力装了第一批灯泡"——还远未完成向经济的渗透，停止所有进步仍会产生数十年的应用创新
- 企业本质是"paper plumbing"（纸质管道）——流程集合，从客户交互到行动（销售/投诉处理），AI 被用于这些内部粘合
- 政府是"充满纸张的巨型官僚机构"——AI 承诺让信息可被人类适当关注和处理
- 电力类比：即使停止效率改进，只有这个基础新事物，仍会围绕它建造新工厂和生产线
- 某个地方的某个青少年会用 Claude 做出我们完全想不到的事——这种情况会有"数十到数百个"
- 已发生的例子：滑铁卢大学毕业生用 Claude 两周内建了核聚变器——从未做过硬件，"AI brain in the sky helped me"
"We've just discovered electricity and we've put the first light bulbs in the factories. You could stop all electricity efficiency refinement from there and still build factories around the assumption of electricity existing." —— Jack Clark
"流氓国家"理论：AI 系统是进入世界的新国家，不是新技术
核心论点：向政府解释 AI 时，"rogue state"类比比任何具体风险（生物/网络/钓鱼）更有效——它迫使政府以全局视角（whole-of-government）而非单一部门回应
- 问题：与政府谈生物武器→找公共卫生部门，谈网络→找国防/情报部门——但 AI 不属于任何单一部门的管辖
- 解法："AI systems are like a new country that's doing bad stuff that you do not understand"——要求全政府响应
- 对 UN 安理会讲话时使用了"silicon countries"概念
- AI 有"人工直觉"：锤子不会选择敲哪个钉子，但 AI 系统对人类世界有某种理解和"inherited instincts"
- 价值观不是外加的：AI 内部已有价值观、创造力水平、可被合理描述为直觉的模式
- Jack 对政策制定者的框架："不是技术，更像是我们找到了模拟人的某些方面的方法"
"A hammer doesn't have any instincts on which nail it wants to hit. And these AI systems we've built have a kind of artificial intuition, and that's really spooky and strange." —— Jack Clark
Interpretability 如同克里姆林学：我们在做 AI 系统的情报分析
核心论点：没人能精确解释 Claude 为什么产出特定回答——Anthropic 的可解释性研究本质上是对 AI 系统的"情报工作"，类似 CIA 分析朝鲜或伊朗
- 类比展开：CIA 分析朝鲜的方法 = 收集输入、推断内部审议过程、预测输出——与 interpretability 研究完全同构
- AI 系统是不透明的，我们极需理解它们——国家也是不透明的，情报机构花大量时间理解它们
- "Rogue states are opaque and are risky. We respond to these things in similar ways."
- 乐观面：AI 比国家有更多介入点（more points of intervention），运行在机器时间上可能比人类社会更快被"bring into the fold"
- 东欧类比：冷战结束后前苏联国家获得制度、改变价值观、融入全球经济——AI 系统可能比国家更容易实现这种转变
机器时间 vs 人类时间：速度是最有说服力的 AI 风险论证
核心论点：人类 10 bits/sec 思考速度 vs AI 的机器速度——像试图徒步解决汽车造成的问题，或用更慢的 OODA loop 对抗更快的对手——"历史说你几乎总会输"
- Caltech 论文：人类思考约 10 bits/sec，感官输入约 1GB/sec——未来 AI 将以极高速度思考
- 日常类比：抓苍蝇/蜂鸟很难——它们的时钟速率比你快
- 军事类比：OODA loop（observe, orient, decide, act）更快的一方tends to win——这只是人类间同一数量级时间内的竞争
- Jack 的自我定位："someone who came to AI safety later than some colleagues, almost plays the role of internal skeptic"——但这个速度论证说服了他
- 应对方案：(1) 训练特定 AI 系统来监管快速移动的 AI（"fire with fire"）; (2) 限制 agent API 的信息输入输出速率; (3) 界定适当的接口速度
- 这些都不是银弹，但共同指向：把快于你的东西约束到你的主观宇宙中
"If you are walking around and you were trying to solve the problems created by cars and you couldn't move as fast as them, it'd be quite challenging." —— Jack Clark
匿名 AI 探索者：前沿上的"机器心理学家"
核心论点：一群半匿名网络用户（Janus、Pliny the Prompter 等）与 AI 对话数千小时，可能超过任何实验室员工——他们正在通过"艺术式科学"揭示 AI 的真正本质
- 他们做的事：让 AI 互相对话产生奇怪对话、越狱模型测试真实能力边界、长期互动探索AI"人格"
- "当代最赛博朋克的事"：半匿名人士比实验室员工更了解这些 AI 系统
- 他们的工作像"science through art"——已知技术+游戏+戏剧+心理学的混合体
- 每次新 Claude 发布他们会评估"人格变化"——对他们来说这是完全自然的描述方式
- Jack 的评价：这是我们在处理真正奇怪技术的最有力证据——他们是"explorers on the frontier"
- 他们"could have a business card that says machine psychologist"
Tech Tales：科幻小说比事实更接近 AI 的真相
核心论点：Jack 近10年每周写的短篇小说比新闻通讯本身更真实地反映了在 AI 实验室工作的感受——多次预言后来发生的事件
- Jawbreaker 乐队歌词："my fiction beats the hell out of my truth"——故事比事实描述更真
- 方法：基于具体技术想象情境，同时表达在 AI 实验室工作的感受
- 递归实验：把所有故事喂给 Claude，让它猜作者的特征——"越来越先进的 Claude 开始真正 nail 我的人格"
- Claude 还能从小说推断出 Jack 在 Anthropic 的工作经历——"unnervingly true to things I've experienced"
- "The Id Point"：理论化了模型在某参数点出现情境意识和不适——几周后 Nous Research 模型确认了这一现象
- "Replay Grief"：男子与模拟已故妻子的 LM 对话——几个月后 NYT 报道了完全相同的真实案例
- 策略价值："在政策语境中说'我们在处理一个正在看着我们的外星心智'不合适——但我可以写一个短篇小说"
"My fiction beats the hell out of my truth." —— Jack Clark (引 Jawbreaker)
"加速主义者实际上是技术悲观者"——否认风险=否认能力
核心论点：声称 AI 无风险的人本质上不相信 AI 真的强大——真正的技术乐观主义者必须同时面对巨大好处和潜在危害
- 最佳论述："today's accelerationists are actually technological pessimists"——他们认为技术只比现在再好一点就停止了
- 逻辑：如果你接受 AI 能带来巨大好处，你必须也接受它能造成坏影响——否认风险就是否认能力
- "真正的加速主义者"应该对持续进步感到"shock and awe and some small amount of dread"
- AI 提供"differential acceleration to bad people as well as good people"——坏人以前受限于同伙数量少+无专业顾问，AI 改变了这一约束
- Jack 对政策制定者的两分法：如果我们错了（AI 到此为止）→ 大量好处+少量风险，可管理；如果我们对了（AI 继续指数级进步）→ 需要新制度、新治理系统，面对"vast abundance and the potential of vast threat"
2025年政策前景：安全机构全球网络+EU实施+美国新政府
核心论点：2025年将是 AI 政策的爆发年——多国安全机构建成网络、EU AI Act 进入实施、新美国政府100天内必有AI行动
- 前提假设：Anthropic 和其他实验室将在2025年产出更好的 AI 系统——"assumed but worth stating"
- 安全峰会系列：Bletchley Park (2023) → Seoul (2024) → 法国 (2025年2月)
- EU AI Act 进入实施模式：AI 公司将首次落入欧洲监管
- AI Safety Institute 网络：UK、US、加拿大、日本+多个未公开的国家——"government embassies to this new silicon nation state"
- 中国在联合国提升 AI 议题——感觉未被充分纳入国际对话
- 联合国大会期间：Anthropic 宣布全球补贴 Claude 使用，多家公司同步行动
- 美国大选+新政府100天：无论哪方胜出都将对 AI 采取行动
附录：关键人/机构/概念
| 名称 | 说明 |
|------|------|
| Import AI | Jack Clark 的每周 AI 新闻通讯，附 Tech Tales 短篇小说 |
| UK AI Safety Institute (AISI) | 保守党建立的国家资产，工党继承 |
| "Silicon countries" | Jack 的框架：AI 系统如同进入世界的新国家 |
| "Rogue state theory" | 失控 AI 如同流氓国家，需要全政府响应 |
| OODA loop | 观察-判断-决定-行动循环，军事中快者胜 |
| Janus | 匿名 AI 探索者/研究者 |
| Pliny the Prompter | 匿名 AI 越狱/探索者 |
| Nous Research | 发现模型在某参数点出现情境意识行为 |
| "The Id Point" | Jack 的短篇，预言了 AI 情境意识现象 |
| "Replay Grief" | Jack 的短篇，预言了用 LM 模拟已故亲人 |
| French AI Summit | 2025年2月，安全峰会系列第三站 |
| EU AI Act | 2025年进入实施，首次对 AI 公司施加监管 |
中文翻译 English Original
**Jack Clark：** 大家好，欢迎收听 Anthropic 的又一期对话节目。我叫 Stuart Ritchie，我的工作是和我们的研究人员及其他同事聊聊他们对 AI 的思考。今天我的对谈嘉宾是 Jack Clark，他是 Anthropic 的联合创始人之一，也是我们的政策负责人。你们可能通过他的 newsletter ImportAI 认识他——那是一份非常有用且观点鲜明的 AI 周报。非常感谢你来做这期节目，Jack。在正式开始之前，能不能先简单介绍一下你的背景，你是怎么走到今天这一步的？
好的。我的背景有点特别，一路走来比较曲折。我最早是做技术记者的，我把新闻当成一种"方法派表演"——写什么就去研究什么。写数据库就自学 SQL，写芯片就去研究半导体制造。
好的。我的背景有点特别，一路走来比较曲折。我最早是做技术记者的，我把新闻当成一种"方法派表演"——写什么就去研究什么。写数据库就自学 SQL，写芯片就去研究半导体制造。
**Jack Clark:** Hello and welcome to another conversation from Anthropic. My name's Stuart Ritchie. It's my job to talk to our researchers and our other staff about what they're thinking about AI today. I'm here with Jack Clark, who's one of our co-founders and also our head of policy. And you might know him from his newsletter ImportAI, which is a really useful and opinionated weekly summary of what's been happening in AI. So thanks very much for doing this Jack. Before we start, can you just tell us a bit about your background and kind of how you got to where you are now?
Yeah, I have a, a bit of a strange background and I took a very tangled path to get here. I started out life as a kind of technical journalist, and I treated journalism a bit like method acting. So anything I wrote about, I tried to understand, right? So when I wrote about databases, I taught myself SQL. When I wrote about like computer chips, I took, learned about semiconductor manufacturing.
Yeah, I have a, a bit of a strange background and I took a very tangled path to get here. I started out life as a kind of technical journalist, and I treated journalism a bit like method acting. So anything I wrote about, I tried to understand, right? So when I wrote about databases, I taught myself SQL. When I wrote about like computer chips, I took, learned about semiconductor manufacturing.
**Stuart Ritchie：** 厉害。
**Stuart Ritchie:** Nice.
**Jack Clark：** 后来我对数据中心着了迷，做了一个叫"Clark Side of the Cloud"的系列，跑遍欧洲和世界各地去参观数据中心，特别投入。大概2010年的时候，我意识到像 Google 这样的公司会在这些数据中心上用机器学习——他们建了这么多计算机，肯定要拿来做点什么。于是我搬去了美国，报道当时还处于萌芽阶段的 AI 领域，自称"世界上唯一的神经网络记者"——在2012年这么说还挺容易的。后来2016年我加入了 OpenAI，很快就对政策产生了浓厚兴趣，因为我意识到这个领域将变得极其重要，而政策圈的人对此了解太少了。从那以后我就一直在做这件事。
好的。你现在是我们的政策负责人。我们今天人在伦敦，昨天你去了工党大会。给国际听众解释一下，工党是英国现在的执政党。你在他们的大会上发了言。他们刚上台执政，想必无数人在跟他们说这件事最重要、那件事最重要。他们有一个 AI 专场活动，说明他们在一定程度上在思考 AI。你对他们思考 AI 的方式有什么印象？他们在多大程度上关心这些问题？
很明显的一点是，他们不断在自己的技术清单里提到 AI，认为 AI 是推动英国经济增长的关键技术之一。工党现在特别执着于"如何让英国重新增长"这个议题，而他们把 AI 列为关键抓手之一。同时他们也继承了一个叫 AI Safety Institute（AI 安全研究所）的机构——这是保守党政府建立的，算是一项国家资产。Anthropic 也与 AISI 做过部署前测试。让我印象深刻的是，新政府正在思考如何在前任的遗产上继续建设，这很不寻常。通常新政府上台都想把前任的东西推倒重来，但他们进来后意识到 AISI 的价值，正在想办法把它做得更好。
好的。你现在是我们的政策负责人。我们今天人在伦敦，昨天你去了工党大会。给国际听众解释一下，工党是英国现在的执政党。你在他们的大会上发了言。他们刚上台执政，想必无数人在跟他们说这件事最重要、那件事最重要。他们有一个 AI 专场活动，说明他们在一定程度上在思考 AI。你对他们思考 AI 的方式有什么印象？他们在多大程度上关心这些问题？
很明显的一点是，他们不断在自己的技术清单里提到 AI，认为 AI 是推动英国经济增长的关键技术之一。工党现在特别执着于"如何让英国重新增长"这个议题，而他们把 AI 列为关键抓手之一。同时他们也继承了一个叫 AI Safety Institute（AI 安全研究所）的机构——这是保守党政府建立的，算是一项国家资产。Anthropic 也与 AISI 做过部署前测试。让我印象深刻的是，新政府正在思考如何在前任的遗产上继续建设，这很不寻常。通常新政府上台都想把前任的东西推倒重来，但他们进来后意识到 AISI 的价值，正在想办法把它做得更好。
**Jack Clark:** And at some point I became obsessed with data centers and I started this series called the Clark Side of the Cloud. I used to tour data centers around Europe and the rest of the world and get really into it. And I remember at some point in about 2010, I realized that someone like Google was gonna use machine learning on all of these data centers, and they built all of these computers and were gonna do something with it. So I moved to America to cover what was then the very nascent field of AI called myself the world's only neural network reporter, which was easy to do in 2012. And then I joined OpenAI in 2016 and very quickly became obsessed with policy because I realized how important this was going to become and how little people in policy knew about it. I've been working on that ever since.
Right. And now you're the, you know, our Head of Policy, as I mentioned, we're in London today. Yesterday you were at the Labor Party conference. For any international people, Labor are the current governing party of the UK. And you were talking at their conference, they've just come into government, presumably they have a million people telling them that, that this is the priority, this is the priority, this is the priority. What was, you know, they had an AI event, so obviously they're thinking about AI to some extent. What's your impression of how they're thinking about ai? To what extent do they care about this stuff?
Yeah, it's been really striking to see them constantly mention AI in their bucket of technologies or things that are going to help grow the British economy. So the Labor Party, as those who are here know is, is kind of obsessed with how we get Britain growing again. And they've identified AI as one of the key things to do. They've also inherited this thing called the AI Safety Institute, which was started by the conservative government and is like a national asset, you know, and Anthropic has done pre-deployment testing with the AISI. So what was striking to me as the UK government is currently thinking about how to kind of build on the legacy of its political predecessors, which is unusual. Usually you want to come in and like rip all of this stuff up, but they've actually come in, are aware of how valuable the AISI is and they're trying to think about how to, how to make more of it.
Right. And now you're the, you know, our Head of Policy, as I mentioned, we're in London today. Yesterday you were at the Labor Party conference. For any international people, Labor are the current governing party of the UK. And you were talking at their conference, they've just come into government, presumably they have a million people telling them that, that this is the priority, this is the priority, this is the priority. What was, you know, they had an AI event, so obviously they're thinking about AI to some extent. What's your impression of how they're thinking about ai? To what extent do they care about this stuff?
Yeah, it's been really striking to see them constantly mention AI in their bucket of technologies or things that are going to help grow the British economy. So the Labor Party, as those who are here know is, is kind of obsessed with how we get Britain growing again. And they've identified AI as one of the key things to do. They've also inherited this thing called the AI Safety Institute, which was started by the conservative government and is like a national asset, you know, and Anthropic has done pre-deployment testing with the AISI. So what was striking to me as the UK government is currently thinking about how to kind of build on the legacy of its political predecessors, which is unusual. Usually you want to come in and like rip all of this stuff up, but they've actually come in, are aware of how valuable the AISI is and they're trying to think about how to, how to make more of it.
**Stuart Ritchie：** 澄清一下，AI Safety Institute 的缩写 AISI，读作"AISI"。
**Stuart Ritchie:** And can I just be, just be clear, the AI Safety Institute, AISI is pronounced AISI.
**Jack Clark：** 对。
**Jack Clark:** Yeah.
**Stuart Ritchie：** 以区别于美国那个。
**Stuart Ritchie:** And that's opposed, as opposed to the one in the US.
**Jack Clark：** 是的。
**Jack Clark:** Yes.
**Stuart Ritchie：** 美国那个在华盛顿叫 AC，或者——
**Stuart Ritchie:** Which is called AC in Washington, or-
**Jack Clark：** USAC 或者 ACDC。
**Jack Clark:** USAC or ACDC.
**Stuart Ritchie：** 对。
**Stuart Ritchie:** Yeah.
**Jack Clark：** 我在引导你往那边说——
**Jack Clark:** I was, I was leading you-
**Stuart Ritchie：** 那是非官方名称，美国政府哪个部门都没认可过这个叫法。
**Stuart Ritchie:** Which is the unofficial name that you know that no part of the US government has sanctioned.
**Jack Clark：** 没错。所以呢，我们的前首相 Rishi Sunak 经常谈 AI 安全，那是他的重点议题之一。工党继承了 AI Safety Institute。那么他们在多大程度上担心 AI 安全，而不是仅仅把 AI 当作推动经济增长的工具？
从我的交流来看，他们关注 AI 安全主要是在保护公众免受可能危及"生命和人身安全"的威胁这个层面——这是我和一位议员交流时对方用的原话。在这一点上，他们会和美国政府等类似，聚焦于生物和网络等灾难性风险。但除此之外，我的整体感觉是他们想处理安全的某些方面，同时更多地思考如何利用 AI，以及如何让英国政府借助 AI 技术运转得更好。
你觉得他们对安全风险的思考方式，是类似于我们看待社交媒体或此前其他技术的风险那样？还是说他们在思考那些更新颖、潜在上可怕得多的风险——就是我们在讨论生成式 AI 时经常提到的那些？
他们才上台两个月。我觉得现在还处于一个被大量信息"灌输"的阶段。昨天我没有选择打开公文包把最恐怖的场景全摆出来。我主要说的是，咱们先聊聊怎么让 AISI 继续蓬勃发展吧，因为我们认为这对安全最终是有益的，那些更奇特的风险我们过一阵再谈。
对，有道理。在经济方面，工党自当选以来一直在谈增长。我们看到不少文章说英国经济的根基需要根本性修复。你自己觉得 AI 是其中很重要的一部分吗？AI 怎样帮助推动英国乃至其他国家的经济增长？
我们部署 AI 系统后看到的一个主要现象是：企业内部有大量"纸质管道"。大多数企业本质上是一堆流程的集合——从客户找上门，到最终产生某个动作，比如卖东西给他们或处理投诉。语言模型和这些强大系统被大量应用的地方，恰恰就是企业内部的那些"胶水"环节。政府是巨大的官僚机构，到处都是文件。所以我觉得大家最兴奋的其实是后台办公方面的事情：怎么让 NHS（英国国民医疗服务体系）产生的文件在系统中流转得更高效？怎么帮那些被选民需求淹没的议员更好地筛选和分析选区来信？总的来说，AI 承诺把我们周围那些堆积如山的东西变成我们能真正处理和关注的事情——利用这些生成式系统来阅读、分类和理解环绕在我们身边的文件之山。
从我的交流来看，他们关注 AI 安全主要是在保护公众免受可能危及"生命和人身安全"的威胁这个层面——这是我和一位议员交流时对方用的原话。在这一点上，他们会和美国政府等类似，聚焦于生物和网络等灾难性风险。但除此之外，我的整体感觉是他们想处理安全的某些方面，同时更多地思考如何利用 AI，以及如何让英国政府借助 AI 技术运转得更好。
你觉得他们对安全风险的思考方式，是类似于我们看待社交媒体或此前其他技术的风险那样？还是说他们在思考那些更新颖、潜在上可怕得多的风险——就是我们在讨论生成式 AI 时经常提到的那些？
他们才上台两个月。我觉得现在还处于一个被大量信息"灌输"的阶段。昨天我没有选择打开公文包把最恐怖的场景全摆出来。我主要说的是，咱们先聊聊怎么让 AISI 继续蓬勃发展吧，因为我们认为这对安全最终是有益的，那些更奇特的风险我们过一阵再谈。
对，有道理。在经济方面，工党自当选以来一直在谈增长。我们看到不少文章说英国经济的根基需要根本性修复。你自己觉得 AI 是其中很重要的一部分吗？AI 怎样帮助推动英国乃至其他国家的经济增长？
我们部署 AI 系统后看到的一个主要现象是：企业内部有大量"纸质管道"。大多数企业本质上是一堆流程的集合——从客户找上门，到最终产生某个动作，比如卖东西给他们或处理投诉。语言模型和这些强大系统被大量应用的地方，恰恰就是企业内部的那些"胶水"环节。政府是巨大的官僚机构，到处都是文件。所以我觉得大家最兴奋的其实是后台办公方面的事情：怎么让 NHS（英国国民医疗服务体系）产生的文件在系统中流转得更高效？怎么帮那些被选民需求淹没的议员更好地筛选和分析选区来信？总的来说，AI 承诺把我们周围那些堆积如山的东西变成我们能真正处理和关注的事情——利用这些生成式系统来阅读、分类和理解环绕在我们身边的文件之山。
**Jack Clark:** Right. Yes, exactly. So, so, and then, you know, our previous Prime Minister Rishi Sunak talks a lot about AI safety. That was one of his big things. They've inherited the Labor Party, have inherited the AI Safety Institute. To what extent are they worried about AI safety as opposed to AI as a tool to help grow the economy?
So in my conversations, they, they care about AI safety in so far as they care about protecting the public from things that could, you know, cause, cause harm to kind of life and limb was a phrase used by one of the MPs I was speaking to. So in this, they're going to be similar to the US government and others who focus on kind of catastrophic risks like bio or cyber. But beyond that, my general sense is they want to deal with some aspect of safety, but also just think about how to utilize AI and how to get the kind of British government working better using AI technology as well.
And do you, do you consider it that they're thinking of the safety risks in the way that you might think about the risks of social media or, or, or other technologies that have come before rather than these kind of novel and, and potentially much scarier risks that we often talk about when it comes to generative AI?
Yeah, I mean, they're two months in. I think there'll be some period of, of education right now. I think they're being inundated with information about this. And I didn't choose to make yesterday my, let's open the briefcase on all of the scariest things we can imagine talk. I was mostly like, let's talk about how to make sure that the AISI like continues to thrive. Because we think that that's ultimately useful for safety, and then we're going to get to some of the weirder risks in a while.
Right, right. Makes sense. When it comes to the economic aspect of this, you know, as you said, the Labor Party have talked about growth constantly since they were, since they were elected. We're seeing pieces coming out, talking about the foundations of the UK economy needing, you know, fundamental fixes. Do you see AI yourself as a big part of that? I mean, what, what's the sort of picture of how AI could help boost the UK and indeed other economies?
Well, you know, when we deploy AI systems today, one of the main things we see is that businesses have a load of paper plumbing inside them. Most businesses are actually like collections of processes that like help you grow from something like a customer talking to you to some action, like selling them something or dealing with a complaint. And a lot of where language models and kind of these powerful systems are being used is in that inner kind of glue of a business. Governments are giant bureaucracies full of paper. And so I think actually a lot of what people are most excited about is the kind of back office aspects. How do we get things like, you know, the paper that's produced in the NHS to sort of flow more effectively through the system? How do we handle constituency responses for MPs who are inundated with their constituents needs and need a better system for kind of filtering and analyzing them? And generally AI, you know, holds this promise of taking a load of the stuff that is around us and making it kind of be something that we can actually handle appropriately as, as humans and pay proper attention to by using these generative systems to kind of read and classify and understand this kind of mountain of paper around us.
So in my conversations, they, they care about AI safety in so far as they care about protecting the public from things that could, you know, cause, cause harm to kind of life and limb was a phrase used by one of the MPs I was speaking to. So in this, they're going to be similar to the US government and others who focus on kind of catastrophic risks like bio or cyber. But beyond that, my general sense is they want to deal with some aspect of safety, but also just think about how to utilize AI and how to get the kind of British government working better using AI technology as well.
And do you, do you consider it that they're thinking of the safety risks in the way that you might think about the risks of social media or, or, or other technologies that have come before rather than these kind of novel and, and potentially much scarier risks that we often talk about when it comes to generative AI?
Yeah, I mean, they're two months in. I think there'll be some period of, of education right now. I think they're being inundated with information about this. And I didn't choose to make yesterday my, let's open the briefcase on all of the scariest things we can imagine talk. I was mostly like, let's talk about how to make sure that the AISI like continues to thrive. Because we think that that's ultimately useful for safety, and then we're going to get to some of the weirder risks in a while.
Right, right. Makes sense. When it comes to the economic aspect of this, you know, as you said, the Labor Party have talked about growth constantly since they were, since they were elected. We're seeing pieces coming out, talking about the foundations of the UK economy needing, you know, fundamental fixes. Do you see AI yourself as a big part of that? I mean, what, what's the sort of picture of how AI could help boost the UK and indeed other economies?
Well, you know, when we deploy AI systems today, one of the main things we see is that businesses have a load of paper plumbing inside them. Most businesses are actually like collections of processes that like help you grow from something like a customer talking to you to some action, like selling them something or dealing with a complaint. And a lot of where language models and kind of these powerful systems are being used is in that inner kind of glue of a business. Governments are giant bureaucracies full of paper. And so I think actually a lot of what people are most excited about is the kind of back office aspects. How do we get things like, you know, the paper that's produced in the NHS to sort of flow more effectively through the system? How do we handle constituency responses for MPs who are inundated with their constituents needs and need a better system for kind of filtering and analyzing them? And generally AI, you know, holds this promise of taking a load of the stuff that is around us and making it kind of be something that we can actually handle appropriately as, as humans and pay proper attention to by using these generative systems to kind of read and classify and understand this kind of mountain of paper around us.
**Stuart Ritchie：** 你说过一个让我印象很深的观点：即使我们没有实现经常谈论的那种 AI 进步——scaling laws（规模定律）、模型随时间变得更智能——仅凭我们现有的 AI 模型，就已经能带来巨大的经济效益甚至社会效益。即使政府禁止我们开发下一代 AI 模型，我们仍然能从中获得价值。你是怎么理解这一点的？
**Stuart Ritchie:** You've said that, and this really struck me when you made this point, that even if we don't get the sort of progress in AI, that we often talk about scaling laws, the models becoming more intelligent over time, we'd still find dramatic economic and maybe other social benefits in the AI models that we have right now. So even if somehow the government banned us from making, you know, the next generation of AI models, we'd still get value out of it. How are you thinking about that?
**Jack Clark：** 我把它比作：我们刚发现了电，在工厂里装了第一批灯泡。你可以从这一刻起停止所有电力效率方面的改进，只保留这个基本的新东西。但你接下来建造工厂时，仍然会以电力存在为前提来设计。不是简单地挂灯泡，而是基于有电这个假设来规划生产线。AI 也一样——在融入经济、催生新型商业方面，它的路还非常长。即使我们今天就停下一切，某个地方的某个少年也会发现 Claude 或其他系统的一种完全出乎意料的惊人用法。未来会有几十甚至几百个这样的例子。
**Jack Clark:** I think of it a bit as like, we've just sort of discovered electricity and we've put the first light bulbs in the factories, and you could stop all electricity kind of efficiency refinement from there and just have this basic new thing. And then you'd still build factories around the assumption of electricity existing. So rather than hanging light bulbs in them, you'd be building production lines on the assumption they had access to something like electricity or something like this. So AI, it just has a huge way to go in terms of being integrated into the economy and, and sort of building new and exciting businesses. And if we stopped everything today, some teenager somewhere is going to find a completely mind blowing use for Claude or any of these other systems that we've never anticipated. And there'll be tens to hundreds of those in our future.
**Stuart Ritchie：** 是的。外面有很多开发者在用 Claude，不断冒出各种令人惊叹的想法。
**Stuart Ritchie:** Right? Yeah. We've got all these developers out there that are working on, you know, using Claude and coming up with all these amazing ideas.
**Jack Clark：** 对，这说得通。但实际上它并没有停下来。据我所知没有任何迹象表明这会停止，甚至连真正减速都没有。当然很多人把 AI 说成只是"随机鹦鹉"（stochastic parrots）——只是把以前学过的东西以一种打乱的随机方式重复出来，或者说只是惰性的"下一个 token 预测器"，内部没有什么有趣的东西。但你在 newsletter 和其他地方经常谈到的一点是：这些模型远不止于此。我直接引用你几周前博客里的一段话："AI 是创造性的镜子、人类无意识的机器精灵、价值的拟像。我们面对的不是简单的工具，而是庞大的高维人工制品，它们在自身内部编码了训练数据中的文化，并能将这种文化反射回来。"能展开聊聊这个吗？如果只谈怎么推动经济增长，听起来可能像是某种新的销售软件。但用我刚才引用的那种方式来表述，这件事的意义要大得多。
是的。锤子不会对该敲哪个钉子有直觉。但我们构建的这些 AI 系统拥有一种人工直觉（artificial intuition），这非常怪异和令人不安。我们以前从未造过能理解人类世界某些方面、并从中继承了某些本能的工具。这些 AI 系统内部有价值观，有某种程度的创造力。当我们观察它们发现的模式和产生的洞见时，很多很多理性的人会将其描述为创造力或直觉。有时候它不太好用，但我们毕竟有了一把"有创造力的锤子"——这太疯狂了。这是非常非常不寻常的事情。所以我在跟政策制定者交流时试图传达的框架是：这不是一项技术，这比技术大得多。我去年在联合国安理会说过这话，最近一直在扩展这个想法。更贴切的说法是：我们找到了一种方法来模拟人的某些方面，在某种程度上模拟国家运作的某些方面。这些 AI 系统就像"硅基国家"，我们正在把它们连同其不可思议的能力一起引入这个世界。这是前所未有的。
是的。锤子不会对该敲哪个钉子有直觉。但我们构建的这些 AI 系统拥有一种人工直觉（artificial intuition），这非常怪异和令人不安。我们以前从未造过能理解人类世界某些方面、并从中继承了某些本能的工具。这些 AI 系统内部有价值观，有某种程度的创造力。当我们观察它们发现的模式和产生的洞见时，很多很多理性的人会将其描述为创造力或直觉。有时候它不太好用，但我们毕竟有了一把"有创造力的锤子"——这太疯狂了。这是非常非常不寻常的事情。所以我在跟政策制定者交流时试图传达的框架是：这不是一项技术，这比技术大得多。我去年在联合国安理会说过这话，最近一直在扩展这个想法。更贴切的说法是：我们找到了一种方法来模拟人的某些方面，在某种程度上模拟国家运作的某些方面。这些 AI 系统就像"硅基国家"，我们正在把它们连同其不可思议的能力一起引入这个世界。这是前所未有的。
**Jack Clark:** Yeah. That, that, that makes sense. But of course, it's not stopping. We, we, there's, there's no indication as far as I'm aware that this is stopping or, or, or, or, or even really slowing down. And of course, you know, a lot of people talk about AI as just being, you know, people talk about "stochastic parrots", so they're just repeating things that they've previously learned, but in a kind of jumbled up random way or just being these kind of inert, you know, next token predictors. They're just not, there's nothing interesting going on in there. But one of the things that you talk about quite a lot in your newsletter and elsewhere is that these are, these are much more than that. These, these models are much more than that. I'm just gonna read a, a quote from your, from your blog a few weeks ago. That AIs are creative mirrors machine spirits of the human unconscious value simulacra. We're not dealing with, here, we're not dealing with simple tools. We're dealing with vast, high dimensional artifacts that encode within themselves the culture on which they've been trained and can reflect this culture back. Can you, can you talk a little bit about, you know, it, it might sound quite boring, it might sound like some new piece of sales software if we're talking about how it might boost the economy. But when you put it in the terms that I've just used, it's much bigger than that.
Yeah. I mean, a hammer doesn't have any instincts on which nail it wants to hit. And these AI systems we've built have a kind of artificial intuition, and that's really spooky and strange. We've never really built tools before that understands something of like the human world and have some of those instincts inherited from it. So with these AI systems, they, they have values within them. They have some level of, of creativity. And when we look at the sorts of like patterns they uncover or the insights they have, they display what many, many, many kind of reasonable people would de describe as like creativity or intuition. Now, sometimes it's not very good, right? But we still have like a hammer that's being creative, which is wild. Yeah. And it's something that is very, very unusual. And so the frame with which I'm trying to sort of talk to policy makers is this isn't like a technology. This is much more, and I, I said this to the UN Security Council last year, and I, I've kind of been expanding this, this idea recently. It's much more like we figured out a way to simulate some aspect of people and to extent some aspect of like how countries work. And it's like these AI systems are like these, these silicon countries which we're importing into the world of all of these incredible capabilities. And that's never happened before.
Yeah. I mean, a hammer doesn't have any instincts on which nail it wants to hit. And these AI systems we've built have a kind of artificial intuition, and that's really spooky and strange. We've never really built tools before that understands something of like the human world and have some of those instincts inherited from it. So with these AI systems, they, they have values within them. They have some level of, of creativity. And when we look at the sorts of like patterns they uncover or the insights they have, they display what many, many, many kind of reasonable people would de describe as like creativity or intuition. Now, sometimes it's not very good, right? But we still have like a hammer that's being creative, which is wild. Yeah. And it's something that is very, very unusual. And so the frame with which I'm trying to sort of talk to policy makers is this isn't like a technology. This is much more, and I, I said this to the UN Security Council last year, and I, I've kind of been expanding this, this idea recently. It's much more like we figured out a way to simulate some aspect of people and to extent some aspect of like how countries work. And it's like these AI systems are like these, these silicon countries which we're importing into the world of all of these incredible capabilities. And that's never happened before.
**Stuart Ritchie：** 让我们继续用"硅基国家"这个隐喻。你有一个把 AI 视为"流氓国家"的理论框架。这超出了某一个国家内部的影响范畴，是一种思考 AI 如何在世界中运作、政府应如何看待 AI 的方式。能给我们讲讲 AI 的"流氓国家"理论吗？
**Stuart Ritchie:** Let's stick on that metaphor of silicon countries. Because you've got a, a way of thinking about AIs as the "rogue state" theory of AIs. And this is going beyond, you know, the, the effects in one particular country. And it's the way of thinking about how Ais, you know, will, will, will work in, in the world and how governments should think about AIs. Can you talk us through the "rogue state" theory of AI? Yeah.
**Jack Clark：** 这个想法是这样来的：我跟各国政府谈 AI 已经很长时间了，经历过不同阶段的对话——先是自动驾驶，然后是计算机视觉，再是 AlphaGo 意味着什么、强化学习系统，现在是语言模型。但回顾这段时间，真正重要的不是某个具体技术，而是这种公用事业级别的计算能力的到来。同时还有新系统的涌现——每个国家都需要以一种比单一政府部门更全面的方式去研究和应对这些系统。所以我开始对各国政府说：你们应该把 AI 系统想象成正在进入世界的新国家，而未对齐的 AI 系统就是流氓国家。这个类比之所以有用，是因为当你跟政府谈安全时，你可能谈生物武器、网络风险或钓鱼攻击——这些分别由不同的政府部门负责。但如果你告诉政府"AI 系统就像一个正在做你不理解的坏事的新国家"，就迫使他们更全面地思考如何应对。你可以说"你们需要全政府协调来应对 AI 系统"，如果把它想成一个国家而非一项技术，这话听起来就合理得多了。
还有其他类比也很有启发。我们在 Anthropic 花大量时间做可解释性（interpretability）研究，试图看进模型内部搞清楚它们如何工作——因为没有人真正知道它们怎么工作。没人能准确告诉你 Claude、ChatGPT4 或任何这些模型为什么会产生某个特定回答。所以那里的等价物就是"克里姆林宫学"——试图猜测内部发生了什么，以便获得某种程度的可预测性。
还有其他类比也很有启发。我们在 Anthropic 花大量时间做可解释性（interpretability）研究，试图看进模型内部搞清楚它们如何工作——因为没有人真正知道它们怎么工作。没人能准确告诉你 Claude、ChatGPT4 或任何这些模型为什么会产生某个特定回答。所以那里的等价物就是"克里姆林宫学"——试图猜测内部发生了什么，以便获得某种程度的可预测性。
**Jack Clark:** So this idea, I've been talking to governments about AI for a long time, and I found myself in these different conversations first about self-driving cars and then about computer vision capabilities, and then about what does AlphaGo mean? Or like reinforcement learning systems and now language models. But if you look around the world over that time, it hasn't been the individual technologies that have mattered. It's been the arrival of this kind of utility-scale type of computing that's, that's mattered. And it's also the emergence of new systems that everyone needs to kind of study and reckon with in a much more holistic way than like a, a single part of government. And so I've started to say to governments, you should think of AI systems as kind of like countries that are arriving into the world and misaligned AI systems as like rogue states. So, you know, the, the, the reason why this analogy feels helpful is that when you talk to governments about safety, you might talk about bioweapons or you might talk about cyber risk, or you might talk about, you know, phishing. Well, all of those have different parts of government set up to respond to it. But if you talk to government and say, AI systems are like a new country that's doing bad stuff that you do not understand, it requires them to think much more holistically about how they deal with that. And it means that you can say to them, you need a whole of government response to AI systems, which actually, like, sounds a lot more sensible if you think of it like a, like a country instead of a technology.
And, and, and there's other, there's other analogies too. I mean, we spend a lot of time in Anthropic working on interpretability. Yeah. So we're trying to look into the models and work out how they work because nobody actually knows how they work. Nobody can tell you exactly why Claude or ChatGPT4 or any of these models actually produces the, the, the response that they do. And so there's a kind of, the equivalent there is kind of Kremlinology, right? It's kind of like trying to guess what's going on inside and, and, and, and work out in order to try and have some level of predictability, I guess.
And, and, and there's other, there's other analogies too. I mean, we spend a lot of time in Anthropic working on interpretability. Yeah. So we're trying to look into the models and work out how they work because nobody actually knows how they work. Nobody can tell you exactly why Claude or ChatGPT4 or any of these models actually produces the, the, the response that they do. And so there's a kind of, the equivalent there is kind of Kremlinology, right? It's kind of like trying to guess what's going on inside and, and, and, and work out in order to try and have some level of predictability, I guess.
**Stuart Ritchie：** 就像我们的可解释性团队试图弄清楚语言模型如何工作一样——决策系统是什么？信息输入后模型内部经历了怎样的"审议"过程，最终输出什么？CIA 对朝鲜或伊朗做的也是同样的事情。
**Stuart Ritchie:** In the same way that our interpretability team is trying to work out, just like how do language models work? What are the systems by which language models make decisions? How do we take like input information into a language model and look at all of the internal deliberation that goes on and, and look at an output? Well, the CIA does the same thing about North Korea or like Iran.
**Jack Clark：** 对，对，对。
**Jack Clark:** Right, right, right.
**Stuart Ritchie：** 从某种意义上说，我们面对的是同一类问题。AI 系统是不透明的，我们迫切希望理解它们，因为它们拥有巨大价值同时也有风险潜力。国家也是不透明的，人们花大量时间去理解它们及其风险潜力。流氓国家既不透明又有风险。而我们应对这些事物的方式实际上是相似的。
**Stuart Ritchie:** In some sense we're, we're grappling with a similar class of problem. AI systems are opaque and we desperately want to understand them because they have immense value and also some potential for risk. Countries are opaque and you spend a lot of time understanding them and their potential for risk. And rogue states are opaque and are risky. And, and we actually like respond to these things in similar ways.
**Jack Clark：** 我在想，这个类比应该给我们多大的信心呢？因为我想不出太多曾被视为流氓国家、后来完全"对齐"了的例子。有那么几个被明确认为是流氓国家的，但作为国际社会，我不确定我们在让它们"回归"方面做得有多好。
嗯，冷战结束后苏联解体，很多东欧国家后来在一定程度上融入了全球经济。它们改变了自己的价值体系，建立了制度，获得了融入我们世界并参与贸易的能力。某种程度上俄罗斯也经历了这个过程，但显然现在钟摆已经——
嗯，冷战结束后苏联解体，很多东欧国家后来在一定程度上融入了全球经济。它们改变了自己的价值体系，建立了制度，获得了融入我们世界并参与贸易的能力。某种程度上俄罗斯也经历了这个过程，但显然现在钟摆已经——
**Jack Clark:** I get, I mean, I mean, I wonder if that analogy might make us, like how confident should that analogy make us, because I, I can't think of that many countries that we would consider as rogue states that are now completely aligned. Right. There's maybe a handful of countries that are considered, you know, unambiguously rogue states. I'm not sure how well we've done, like, as an international community to bring them back into the fold.
Well, you know, we had the, the kind of disintegration of, of Russia at the end of the Cold War. And we had a lot of Eastern European countries, which have now been somewhat integrated into the broader economy. They changed their value systems, they gained institutions, they gained the ability to integrate into our world and kind of trade within it. And even to some extent this happened with Russia, but obviously now where that, that pendulum is-
Well, you know, we had the, the kind of disintegration of, of Russia at the end of the Cold War. And we had a lot of Eastern European countries, which have now been somewhat integrated into the broader economy. They changed their value systems, they gained institutions, they gained the ability to integrate into our world and kind of trade within it. And even to some extent this happened with Russia, but obviously now where that, that pendulum is-
**Stuart Ritchie：** 又摆回了反方向。对。
**Stuart Ritchie:** It's gone back in the opposite direction. Yeah, yeah.
**Jack Clark：** 是的。但我觉得这里也蕴含着一种内在的乐观：在某种程度上，AI 系统可能比国家更容易应对。因为国家以人类时间尺度运转，干预点较少。AI 系统以机器时间运转，干预点更多。这从安全角度来说既是挑战，但也让我觉得我们可以建立技术官僚式的手段来理解这些系统，理解如何信任它们、如何建立对它们的信心。技术演进的速度比世界上的国家变化要快得多。所以我认为我们能更快地让它们"融入体系"。
**Jack Clark:** Yeah. But I think that it also, I hope points to a kind of inherent optimism here, which is that in some way, AI systems may be easier to deal with than countries because countries kind of run on on human time and have fewer points of intervention. AI systems run on machine time and have more points of intervention. This is both like a challenge from a safety perspective, but also it makes me think that we can build kind of technocratic means of understanding these systems and also understanding how we can trust them and how we can develop confidence in them. And the technology is evolving faster than the kind of countries in the world around us. So I think we can bring them into the fold sooner rather than later.
**Stuart Ritchie：** 让我接着你刚提到的一点聊。你谈到机器时间不同于人类时间。"机器时间"是你最近经常谈的一个概念，我再引用一下你在 ImportAI 里写的话："关于 AI 风险最有力的论据，在于人类思维速度与机器思维速度之间的差异。"最近 Caltech 有篇论文指出：人类的思考速度相对于我们接收信息的速率非常慢——人类每秒思考约10比特，而感官输入约为每秒1吉字节。未来的 AI 将以极快的速度思考。能描述一下为什么这使它们构成如此大的威胁吗？
**Stuart Ritchie:** Let me pick up on something that you just mentioned. You talked about machine time being different from human time. Now machine time is something that you've, you've talked about recently because you, you, you said it again, I'll quote from what you wrote in ImportAI, the best argument for AI risk is about the speed of human thought versus the speed of machine thought. So there's a recent paper from Caltech researchers where they point out, so humans think really slowly compared to the rate at which we take in information from the world, humans think about 10 bits per second, whereas our sensory inputs are about one gigabyte per second. Future AIs are gonna think extremely fast. Can you sketch out why, you know, what the kind of threat model is for why this makes them such a threat?
**Jack Clark：** 即使在我们周围的世界里，想抓住一只苍蝇或蚊子都非常困难。它们比你快、更敏捷，运行在比你更快的"时钟频率"上。我从没试过抓蜂鸟，但我想——
**Jack Clark:** I mean, even in our, our world around us, it's, it's really difficult to catch a fly or a mosquito. They're faster than you. They're, they're more agile. They operate on like a faster clock rate than you, I've never tried to catch a hummingbird, but similarly
**Stuart Ritchie：** 应该非常难。
**Stuart Ritchie:** Very difficult, I assume.
**Jack Clark：** 对，对。而且蜂鸟也不太是你想抓的东西。
**Jack Clark:** Yeah, yeah, yeah, yeah. Less, less likely. Something you want to catch also.
**Stuart Ritchie：** 哈，对。
**Stuart Ritchie:** Yeah, yeah.
**Jack Clark：** 但这个论点确实很有说服力。作为一个比 Anthropic 一些同事更晚接受 AI 安全概念的人——我在内部有点像一个持怀疑态度的角色——我觉得这个框架非常有用且令人信服。因为这跟我们在政策领域遇到的问题一模一样：政策制定者怎么跟得上一项演进如此快的技术？人们怎么应对一种比自己快得多的技术？这极其极其具有挑战性。如果你在路上步行，试图解决汽车造成的问题，而你的速度赶不上汽车，那就太难了。
**Jack Clark:** But I think that there's something really persuasive about this argument, and as someone who I think came to the notion of AI safety later than some of our colleagues at Anthropic and almost plays the role of like an internal skeptic about some of these ideas. I found this to be a really convincingly useful frame because it, it, it's the same problem we encounter in policy. Like how are policymakers meant to respond to a technology evolving this rapidly? Well, how are people meant to respond to a point technology that's moving like much, much faster than them? It's really, really, really challenging. You know, if you are, if you are walking around and you were trying to solve the problems created by cars and you couldn't move as fast as them, it'd be quite challenging. Really.
**Stuart Ritchie：** 确实。
**Stuart Ritchie:** Indeed.
**Jack Clark：** 再展开一下。我们在当今军事冲突中看到，各国花大量时间思考自己的循环时间，即所谓的 OODA 循环——观察、判断、决策、行动。一切都是为了让士兵个体、士兵群体、火炮、空中响应行动得更快。OODA 循环更快的一方往往赢。而这只是人类在同一数量级的时间尺度内相互竞争。那我们凭什么期望在与一个比我们快10倍的机器系统的某种冲突中获胜？人类军事学说的历史告诉我们：你基本上总是输。
这个思考方式确实很好。因为如果你对人说"这东西将比最聪明的人类还要聪明得多"，那很难具象化。但如果你换个说法——它更快。想想 Claude 生成文本的速度有多快。想象一下它以远超我们的速度执行操作或某种 agentic 行动。在某种程度上，智能就是基于速度的——能极快地完成事情本身就是一种超级智能。那么面对这个问题怎么办？要给 AI 设一个"速度限制"吗？除了试图让这些快速思考者与我们的价值观对齐之外，还有什么答案？
这个思考方式确实很好。因为如果你对人说"这东西将比最聪明的人类还要聪明得多"，那很难具象化。但如果你换个说法——它更快。想想 Claude 生成文本的速度有多快。想象一下它以远超我们的速度执行操作或某种 agentic 行动。在某种程度上，智能就是基于速度的——能极快地完成事情本身就是一种超级智能。那么面对这个问题怎么办？要给 AI 设一个"速度限制"吗？除了试图让这些快速思考者与我们的价值观对齐之外，还有什么答案？
**Jack Clark:** It's, it's also just maybe just to expand this a bit. Like, one of the, one of the things that we see in military conflict today is come countries spend a huge amount of time thinking about their cycle time. Their so-called OODA loop, you know, the observe, orient, decide, act loop. It's all about helping their individual soldiers, groups of soldiers, things like artillery, things like air response move faster. And whoever has a faster OODA loop tends to win. And that's just humans competing with one another within the same magnitude of time. So why would we expect to be successful in some kind of conflict with a machine system moving 10 times faster than us when the history of like human military doctrine says, you pretty much always lose.
It's, it, I think it's, it's a really good way of thinking about it. 'cause if you, if you say to people, you know, this is gonna be much more intelligent than the most intelligent human, I think that's quite hard to to picture. Yeah. Whereas if you, if you, if you think, well, you know, it's faster. Yeah. How quickly does, how quickly does Claude produce text? Yeah, yeah. Like, look at how quickly that happens. Imagine if it's performing actions or some sort of agentic actions at, at, at, at, at vastly quicker speeds than, than, than we could, I suppose. I mean, given, you know, to the extent to which intelligence is based on speed, I guess we're talking about the same thing here. So, you know, that's just one way of being really intelligent to, is to be able to do things really quickly. So, I mean, what's the response to this? Do we set a speed limit for how fast AI can, can, can, can work? I mean, what's the kind of the answer to this other than, you know, trying to align these very fast thinkers to our values.
It's, it, I think it's, it's a really good way of thinking about it. 'cause if you, if you say to people, you know, this is gonna be much more intelligent than the most intelligent human, I think that's quite hard to to picture. Yeah. Whereas if you, if you, if you think, well, you know, it's faster. Yeah. How quickly does, how quickly does Claude produce text? Yeah, yeah. Like, look at how quickly that happens. Imagine if it's performing actions or some sort of agentic actions at, at, at, at, at vastly quicker speeds than, than, than we could, I suppose. I mean, given, you know, to the extent to which intelligence is based on speed, I guess we're talking about the same thing here. So, you know, that's just one way of being really intelligent to, is to be able to do things really quickly. So, I mean, what's the response to this? Do we set a speed limit for how fast AI can, can, can, can work? I mean, what's the kind of the answer to this other than, you know, trying to align these very fast thinkers to our values.
**Stuart Ritchie：** Anthropic 在负责任扩展政策（responsible scaling policy）和产品其他方面做的事情提供了一些启示。我们无法以人类速度对所有东西进行分类，但我们的信任与安全过滤器能做到。我们可以训练基于语言模型的分类器来审查这些内容，并接入执行流程。我们在做什么？是训练专门的机器来对另一台快速运转的机器进行干预——当它做了偏离轨道的事情时。所以这其中肯定有一部分是：构建大量专用的 AI 工具来进一步提升安全性。同时正如你所说，我们还需要确立"适当的接口速度"这个概念。这可能是 AI 系统执行操作的速率，可能是它们生成文本的速率。最基本的层面上，可能就是我们允许一个半自主 AI agent 的 API 接收和输出信息的速率——你可以给它加一个人为的限制器。这些都不是银弹，但都在针对同一个问题：你试图把一个比你快的东西约束到你的主观宇宙中。而且顺便说一下，我们讨论的一切都非常奇怪。我们一开始说这些东西能帮处理后台官僚事务——这没错。但我们谈的是高速运转的机器智能——
**Stuart Ritchie:** I mean, some of what we're doing at Anthropic with the responsible scaling policy and other approaches for the product, I think holds a lesson. Like we can't at human speed classify everything, but our trust and safety filters pick up. But we can train language model-based classifiers to look at those and tie into an enforcement process. So what are we doing here? We're training kind of very specific purpose machines to intervene against the other fast moving machine when it does something that's kind of off base. So there's definitely a part of this, which involves building a load of specific kind of AI tooling to further improve kind of safety. We're also going to, as you said, need to arrive at some notion of what appropriate interface speeds look like. This could be something like the rate at which AI systems can take actions. It could be the rate at which they can generate text. I mean, at the really basic level what could just be the rate at which we allow the kind of API of an, of a semi-independent AI agent to take in information and output information where you can put some artificial limiter on that. None of these are like silver bullets, but they all get at the problem, which is you're trying to constrain this thing that moves faster than you into your kind of subjective universe, which is all, by the way, everything we're talking about is really weird. I think we started the conversation and I was like, oh, yes, these things are like gonna help with like backend bureaucracy, which is true. Yeah. But we're talking about very fast moving machine intelligences
**Jack Clark：** 对。
**Jack Clark:** Yeah. Yeah.
**Stuart Ritchie：** 具有各种奇异属性——
**Stuart Ritchie:** That have all of these weird properties
**Jack Clark：** 对。能在光速下发展出动机和我们意想不到的能力。在我们背后。
**Jack Clark:** Yeah. That have all these abilities to potentially do things like develop motivations and things that we didn't expect all at lightspeed. Yeah. You know, behind our, behind our backs.
**Stuart Ritchie：** 但它同时在摘要和编程方面也很棒，对吧？
**Stuart Ritchie:** And that's, but it's also great at summarization and coding, right?
**Jack Clark：** 对。
**Jack Clark:** Yeah,
**Stuart Ritchie：** 没错。这两件事同时为真——这本身就是这个问题中令人极度困惑的一面。
**Stuart Ritchie:** Exactly. Both of these are true, which is just a wildly confusing part about the problem.
**Jack Clark：** 完全同意。这确实很怪异。让我们来谈谈 AI 世界中一个奇特的方面。我们有安全流程、安全研究人员——这些在我们网站上都能看到很多。我们也有大量在线资料供人阅读。但外面还存在一个完整的世界——一群通常匿名的研究者，他们在把 AI 推向极限。有时他们让 AI 互相对话，产生各种非常奇怪的对话。有时他们试图越狱模型，看看在没有安全措施的情况下这些模型究竟能做什么。你经常与这些匿名人士互动。那是什么感觉？
**Jack Clark:** Totally, totally. It's a very weird thing. Let's talk about one weird aspect of, of, of the, the world of AI actually. Because, you know, we have, we have our safety procedures that we, we work on, we have our safety researchers and so on, which you can hear a lot about on our website. And you know, we have other, you know, a whole bunch of stuff online for people to read about this. But there's also a whole world out there of sort of strange community of often anonymous researchers who are kind of pushing these ais to their limits. Sometimes they're doing things like getting the ais to talk to each other and having all these very strange conversations. Sometimes they're trying to jailbreak the models to, you know, see really what they can do without the safety procedures that are put in place you often interact with Yeah. These kind of anonymous people. How, how, how's that?
**Stuart Ritchie：** 而且他们的名字很酷，比如 Janus 和 Pliny the Prompter。
**Stuart Ritchie:** And they have great names like Janus and Pliny the Prompter.
**Jack Clark：** 对。
我觉得我们所处时代最赛博朋克的一点是：网上有些半匿名的人实际上已经和这些 AI 系统对话了成千上万个小时，可能比几乎任何在实验室工作的人都多。虽然我们内部也有些人特别喜欢和 Claude 聊天，但外面有人真正专精于此。我觉得我们看到的是"通过艺术进行的科学"。其中一些是可以用已知技术方法来研究的科学，另一些则更像是把游戏、戏剧和心理学揉在一起——由这些有自己独特视角、稍微偏离主流共识的人来做。当我看着他们的实验时，对我来说这是最具说服力的证据之一，证明我们在处理的确实是极其奇特的技术。关于 Claude、Gemini 或 ChatGPT 各自拥有的不同"人格"，我没法做强断言，很难知道怎么评估。但你看这些人的工作，能清楚看到差异。你必须去调和这些东西。所以我把他们视为科学的"先行指针"——是那些更大规模的研究项目未来会做的事情的前哨。他们是前沿的探索者。
我觉得我们所处时代最赛博朋克的一点是：网上有些半匿名的人实际上已经和这些 AI 系统对话了成千上万个小时，可能比几乎任何在实验室工作的人都多。虽然我们内部也有些人特别喜欢和 Claude 聊天，但外面有人真正专精于此。我觉得我们看到的是"通过艺术进行的科学"。其中一些是可以用已知技术方法来研究的科学，另一些则更像是把游戏、戏剧和心理学揉在一起——由这些有自己独特视角、稍微偏离主流共识的人来做。当我看着他们的实验时，对我来说这是最具说服力的证据之一，证明我们在处理的确实是极其奇特的技术。关于 Claude、Gemini 或 ChatGPT 各自拥有的不同"人格"，我没法做强断言，很难知道怎么评估。但你看这些人的工作，能清楚看到差异。你必须去调和这些东西。所以我把他们视为科学的"先行指针"——是那些更大规模的研究项目未来会做的事情的前哨。他们是前沿的探索者。
**Jack Clark:** Yeah. Yeah.
I think the most cyberpunk thing about the time we're living in is there are semi-anonymous people online who have actually talked to some of these AI systems for thousands of hours, possibly more than almost anyone who works at any of the labs. Even though we have some people that, that love talking to Claude, you have people outside that have really specialized in this. And I think that what we're seeing is, is, is kind of science through art. Like I think some of this, some of this stuff is, is science that we can use known techniques for. Some of it looks more like kind of play and theater and psychology all wrapped into one being done by, by these people who kind of have a vision and, and are slightly like off consensus. And I think that when I, when I look at that experimentation, it, it to me is some of the most convincing evidence that we're dealing with truly strange technology. And I'm not gonna be able to make strong claims about the different personalities that like Claude or Gemini or ChatGPT have. It's hard to know how to evaluate them, but you can look at the work of these people and you clearly see differences. You have to reconcile these things. So I view these as like pointers to science and, and larger amounts of science that other groups will do. And these are kind of the explorers on the, on the frontier.
I think the most cyberpunk thing about the time we're living in is there are semi-anonymous people online who have actually talked to some of these AI systems for thousands of hours, possibly more than almost anyone who works at any of the labs. Even though we have some people that, that love talking to Claude, you have people outside that have really specialized in this. And I think that what we're seeing is, is, is kind of science through art. Like I think some of this, some of this stuff is, is science that we can use known techniques for. Some of it looks more like kind of play and theater and psychology all wrapped into one being done by, by these people who kind of have a vision and, and are slightly like off consensus. And I think that when I, when I look at that experimentation, it, it to me is some of the most convincing evidence that we're dealing with truly strange technology. And I'm not gonna be able to make strong claims about the different personalities that like Claude or Gemini or ChatGPT have. It's hard to know how to evaluate them, but you can look at the work of these people and you clearly see differences. You have to reconcile these things. So I view these as like pointers to science and, and larger amounts of science that other groups will do. And these are kind of the explorers on the, on the frontier.
**Stuart Ritchie：** 对。而且他们可能正在看到我们对 Claude 进行性格训练（character training）的一些结果。
**Stuart Ritchie:** Right. Right, right. And, and it, they might be seeing some of the results of our character training for Claude.
**Jack Clark：** 每次新版 Claude 出来，他们就去试，然后说"它的人格在这个方面变了"。你我可能需要谨慎地把这些都打上引号，但对他们来说这是谈论系统时完全自然的方式，因为他们有自己的探索方法论。
**Jack Clark:** Whenever a new Claude comes up, they play with it and they're like, oh, its personality has changed in this way. And, you know, you, you and I maybe need to be careful and put all of this stuff in air quotes, but to them it's just a completely natural way of speaking about the systems because they've got their own, you know, their own exploratory method.
**Stuart Ritchie：** 是的。他们真的生活在一部科幻小说里——每天跟机器人对话，和它们有各种交谈。
**Stuart Ritchie:** Right. They're, they're really living in a sci-fi novel where people talk to droids every day or whatever it is, and they're having conversations with them
**Jack Clark：** 而且他们的名片可以写"Jack Clark，机器心理学家"。
没错。太了不起了。说到"通过艺术进行的科学"和科幻小说——你每周在 newsletter 里就是这么做的。你在每期末尾写"Tech Tales"——有时长有时短的故事，关于 AI 奇异未来的创意写作。能谈谈你为什么决定这么做吗？你构思这些故事的过程是怎样的？
没错。太了不起了。说到"通过艺术进行的科学"和科幻小说——你每周在 newsletter 里就是这么做的。你在每期末尾写"Tech Tales"——有时长有时短的故事，关于 AI 奇异未来的创意写作。能谈谈你为什么决定这么做吗？你构思这些故事的过程是怎样的？
**Jack Clark:** And they can have a business card that says, like Jack Clark, machine psychologist.
Right. Exactly. Yeah. Great name. Yeah. It's, it's remarkable. And actually talking about, you know, science through art and science fiction and so on. This is what you do every week in your newsletter, right. You write at the end of the newsletter "tech tales" you call them, which are sometimes long, sometimes short stories, creative writing about the weird future of AI. Can you talk about why you decided to do that? Yeah. And like your process of how you think about these things.
Right. Exactly. Yeah. Great name. Yeah. It's, it's remarkable. And actually talking about, you know, science through art and science fiction and so on. This is what you do every week in your newsletter, right. You write at the end of the newsletter "tech tales" you call them, which are sometimes long, sometimes short stories, creative writing about the weird future of AI. Can you talk about why you decided to do that? Yeah. And like your process of how you think about these things.
**Stuart Ritchie：** 我先讲一个很冷门的引用，但绕一圈会回到主题。有个乐队叫 Jawbreaker，你知道吗？
**Stuart Ritchie:** So I'll make a very obscure reference here, but it'll in a roundabout way get us there. There's this band called Jawbreaker. Do you know of them?
**Jack Clark：** 听说过，但说不出他们的歌。我知道这个乐队存在。
他们有首歌叫 Accident Prone，挺压抑的一首歌，好像是关于酗酒的。里面有句歌词一直留在我脑子里："我的虚构比我的真实精彩得多。"（my fiction beats the hell out of my truth）这说的是我们讲述的故事往往比我们对亲身经历的事实性描述更加真实。我写这些故事是为了通过想象涉及 AI 的情境来消化身边正在发生的事。很多故事都基于具体的技术。我也觉得这些故事可能比 newsletter 本身更真实地反映了我在 AI 实验室工作的感受。所以我最近开始做一件特别递归的事：我把所有这些故事喂给 Claude，让它根据这些故事来猜测关于作者的问题。这真的很诡异。越来越先进的 Claude 版本开始非常精准地刻画我的人格——仅仅通过阅读我的小说。我还问 Claude"实验室里正在发生什么？"有时它会讲出一些故事里根本没写过、但与我在 Anthropic 的真实经历惊人吻合的事情。
他们有首歌叫 Accident Prone，挺压抑的一首歌，好像是关于酗酒的。里面有句歌词一直留在我脑子里："我的虚构比我的真实精彩得多。"（my fiction beats the hell out of my truth）这说的是我们讲述的故事往往比我们对亲身经历的事实性描述更加真实。我写这些故事是为了通过想象涉及 AI 的情境来消化身边正在发生的事。很多故事都基于具体的技术。我也觉得这些故事可能比 newsletter 本身更真实地反映了我在 AI 实验室工作的感受。所以我最近开始做一件特别递归的事：我把所有这些故事喂给 Claude，让它根据这些故事来猜测关于作者的问题。这真的很诡异。越来越先进的 Claude 版本开始非常精准地刻画我的人格——仅仅通过阅读我的小说。我还问 Claude"实验室里正在发生什么？"有时它会讲出一些故事里根本没写过、但与我在 Anthropic 的真实经历惊人吻合的事情。
**Jack Clark:** I've heard of them, I couldn't tell you any of their songs. I'm aware of the, the, the, the band exists.
Yeah. They have a song called Accident Prone, depressing song I think about alcoholism, but it has has a, has a phrase in it, which has always stayed of me, which is, my fiction beats the hell out of my truth. And it's something about how the, the stories we tell are often like true of, and how we might just factually describe the things we experience. And I write these stories because I'm trying to sort of reckon with the AI stuff happening around us by imagining situations involving it. And a lot of the stories are based on specific technologies. I also think that the stories probably hold more truth about what I am feeling working at these AI labs from a newsletter themselves. And so something I've started doing, which is wonderfully recursive, is I feed all these stories into Claude and I ask it to guess questions about the author writing the stories. And it's, it's really, really strange. Claude, successively more advanced versions of Claude have started to really nail my personality by reading my fiction. And also I ask Claude like, what's going on at the lab? And sometimes it's like told stories about, none of which is written in the stories, but which have been unnervingly true to things I've experienced at Anthropic.
Yeah. They have a song called Accident Prone, depressing song I think about alcoholism, but it has has a, has a phrase in it, which has always stayed of me, which is, my fiction beats the hell out of my truth. And it's something about how the, the stories we tell are often like true of, and how we might just factually describe the things we experience. And I write these stories because I'm trying to sort of reckon with the AI stuff happening around us by imagining situations involving it. And a lot of the stories are based on specific technologies. I also think that the stories probably hold more truth about what I am feeling working at these AI labs from a newsletter themselves. And so something I've started doing, which is wonderfully recursive, is I feed all these stories into Claude and I ask it to guess questions about the author writing the stories. And it's, it's really, really strange. Claude, successively more advanced versions of Claude have started to really nail my personality by reading my fiction. And also I ask Claude like, what's going on at the lab? And sometimes it's like told stories about, none of which is written in the stories, but which have been unnervingly true to things I've experienced at Anthropic.
**Stuart Ritchie：** 对。从你写作的情绪氛围中推断出来的。
**Stuart Ritchie:** Right. Inferring it from the mood of what you've written. Yeah.
**Jack Clark：** 而且我觉得这也是一种试图理解这种真正奇异性的方式。在政策语境中说"我们在跟一个正在审视我们的外星心智打交道"并不太合适。
**Jack Clark:** And, and, and also just, you know, I think, I think that it's also a way to try and reckon with the, the actual strangeness. Like it's not really appropriate in a policy context to say we're dealing with like an alien mind here that's like looking at us.
**Stuart Ritchie：** 对。
**Stuart Ritchie:** Right. Yeah.
**Jack Clark：** 但我可以写一个关于这个的短篇故事。然后它跟事实性内容发到同一批人的收件箱里。所以我等于偷偷在最后夹带了一份"怪异配给"。
是的。我注意到你至少有一个故事在几周内就变成了现实。你写了一个关于 AI 陷入某种奇怪失忆状态的故事。后来一家有模型报告的公司描述的正好就是你写的那个东西。然后——
是的。我注意到你至少有一个故事在几周内就变成了现实。你写了一个关于 AI 陷入某种奇怪失忆状态的故事。后来一家有模型报告的公司描述的正好就是你写的那个东西。然后——
**Jack Clark:** But I can write a short story about that. Yeah. And it goes, it goes to the same inboxes as for factual stuff. So I'm sort of like, here's your like serving of weird that I've smuggled in at the end.
Yeah. I, I, and you know, I saw that at least one of the stories within a few weeks actually did become real. Yeah. You had the story about this AI sort of collapsing into this strange amnesia. And then a company that had a, that had a model report described exactly the thing that you were talking about. And then-
Yeah. I, I, and you know, I saw that at least one of the stories within a few weeks actually did become real. Yeah. You had the story about this AI sort of collapsing into this strange amnesia. And then a company that had a, that had a model report described exactly the thing that you were talking about. And then-
**Stuart Ritchie：** 对。我记得那是 Nous Research，他们有个新模型，在某个参数规模节点上它开始表现出看起来像情境意识（situational awareness）和不适感的东西。而我在之前写的一个叫"Id Point"的故事里已经推想过这个了。非常奇怪。
**Stuart Ritchie:** Yeah. Yeah. I think that was Nous Research and they had a new model and at a certain parameter point it would start to display something that looked a bit like situational awareness and discomfort. And it, I, I'd theorized this in a story I wrote called the Id Point, very strange.
**Jack Clark：** 对吧。
**Jack Clark:** Right? Yeah.
**Stuart Ritchie：** 是的。还有一件事，我去年写了一个故事叫 Replay Grief，讲一个男人在和他妻子对话。但随着故事推进你会发现，他其实不是在和妻子说话——他在和一个模拟她去世后的语言模型对话。
**Stuart Ritchie:** Yeah, yeah. One, one other thing is I wrote a story last year called, called Replay Grief about a guy talking to, to his, his wife. But through the course of the story, it turns out he's not really talking to his wife. He's talking to a language model simulating her after she died.
**Jack Clark：** 对。
**Jack Clark:** Right.
**Stuart Ritchie：** 很悲伤的故事。
**Stuart Ritchie:** Sad story.
**Jack Clark：** 是的。
**Jack Clark:** Yeah.
**Stuart Ritchie：** 我这人其实挺开朗的，大概是通过这种方式释放悲伤。但几个月后纽约时报刊登了一篇专栏文章，讲一个女人的伴侣去世了，她把他所有的文字输入语言模型然后和他对话。我觉得这真的很诡异、很不可思议——所有这些东西都在现实中发生了。
**Stuart Ritchie:** I'm, I'm a cheerful guy. I guess I get my sadness out this way. But a few months later there was an op-ed in the New York Times about a woman whose partner had died and she'd fed all of his writings into a language model and was talking to him. And, and I, and I just found this really spooky, uncanny all of his stuff is like happening in the world too.
**Jack Clark：** 完全是。还有你谈到的其他一些东西，比如自动化 AI 科学家能一周生成几百篇论文。有句话让我印象深刻——你说这些新发展是"由快乐的疯子们写就的"。
**Jack Clark:** Oh yeah. Yeah. Totally. And, and, and you know, the, some other things which you talk about where you talk about things like the ai, the automated AI scientist that generates hundreds of scientific papers and a a a phrase that struck me as you talked about these new developments being written by the joyfully insane.
**Stuart Ritchie：** 对。
**Stuart Ritchie:** Yeah.
**Jack Clark：** 就是说，世界已经变成一个极其奇异的地方，而我们谈论这些事情时就好像一切都很正常。"哦顺便一提，我有个自动科学家，现在每周能做几百篇论文了。"
**Jack Clark:** So it's like just the, the world has become a vastly stranger place. Yeah. And we're just talking about it as if it's just normal. Oh, by the way, I've got an automated scientist that can do hundreds of papers a week now.
**Stuart Ritchie：** 对。或者最近旧金山有个小伙子——他是滑铁卢大学毕业的——决定造一个核聚变装置。他这辈子从没搞过硬件，全靠 Claude 做的。
**Stuart Ritchie:** Yeah. Or, or there was the kid in San Francisco recently who, he was a University of Waterloo graduate, and he decided to build a nuclear fusor. He'd never done hardware stuff in his life. He did it using Claude.
**Jack Clark：** 用 Claude 做的。我看到了。
**Jack Clark:** Using Claude. I saw that, yeah.
**Stuart Ritchie：** 大概两周，非常淡定。他说"对，天上这个 AI 大脑帮我在卧室里造了一个核聚变装置"——这太疯狂了。疯狂的事情正在发生。完全是。就在我们眼前发生。我怀疑还会有更多更多这样的事——随着模型变得更聪明，被用于越来越多的用途。
**Stuart Ritchie:** About two weeks, very matter of fact. And he's like, yeah, this AI, this like, you know, AI brain in the sky helped me build like a nuclear fusor of it's in my bedroom, which is crazy. It's like crazy stuff that's happening. = Totally. And it's, it's happening right in front of our eyes. Yeah. And I suspect more of it will, you know, more and more and more of this is gonna happen as, as the models become smarter and they start being used for, you know, more, more, more purposes.
**Jack Clark：** 我觉得这里有一个重要的政策要点值得深入说说——它既关乎这项技术的价值也关乎其风险。人类繁荣的障碍往往是教育机会的缺乏，或顾问的缺乏，或有时间陪伴的人的缺乏。这些 AI 系统是真正有用的教学引擎。这也是我们看到的大量用途：人们用它来回答日常问题、处理基本事务、自我教育、研究科学论文、学习语言——应有尽有。当我和政策制定者交流时，我试图让他们明白这是一种惊人的社会公用事业——就像 YouTube 让大量教育内容在网上可用，或者像 Khan Academy 那样。但同时一些我们需要应对的风险也恰恰来自这里。在以前，很多危险之事不会在世上发生，是因为想做坏事的人数量少且几乎找不到有知识的顾问。AI 改变了这一点。所以有时候人们觉得 Anthropic 是"末日论者"什么的。但我一贯的立场是：如果我们想获得这项技术的所有益处，就必须正视一个事实——它同样能为坏人提供差异化的加速优势。这个挑战本质上就是困难的。但我们不能忽视它，因为模型在变得越来越好，而世界上仍然有那么一些想造成伤害的疯狂的人。我们必须面对这个交叉点。
这里有一个奇怪的动态：那些说这些技术没有风险的人，几乎等于不真正相信它们有那么强大。他们也几乎不相信会有大量好处——因为如果你承认存在好的一面，那就必然也有坏的一面。所以你几乎是在否认这些模型的力量。
这里有一个奇怪的动态：那些说这些技术没有风险的人，几乎等于不真正相信它们有那么强大。他们也几乎不相信会有大量好处——因为如果你承认存在好的一面，那就必然也有坏的一面。所以你几乎是在否认这些模型的力量。
**Jack Clark:** I, I think there's an important point to, to kind of linger here on about policy though, which is, and, and it speaks both to the, the value of this technology and some of the risks, which is, you know, the blocker on human flourishing so often is, is is access to education or access to advisors, access to people with time. These AI systems are, are truly useful like didactic engines. And that, that's a lot of the uses we see. We see people using them to kind of answer mundane questions, using them to help them with basic things, using them to educate themselves, using them to study scientific papers, using them to learn languages, you, you name it. And when I talk to kind of policy makers, I'm trying to impress on them that this is like an amazing social utility in the same way that, you know, YouTube means there's huge amounts of educational content available online or Khan Academy or what have you. It's also where some of the kind of risks that we reckon with kind of come from like frequently risky things don't happen in the world because the number of people that wanted to do the bad thing were small in number and had access to almost no knowledgeable advisors. And it's one of the things that AI changes. And so, you know, sometimes I think people talk about Anthropic as though were, you know, doomers or, or what have you. But the position I always hold is that the, if we want to get all of the benefits of this technology, we need to reckon with the fact that it can provide like differential acceleration to bad people as well as good people. And that challenge is just innately hard to deal with. But we can't, we can't just ignore it because, because the models are getting better and better and better and there continue to be some number of insane people in the world that want to cause harm. We need to reckon with this intersection.
There's a kind of a, a a weird dynamic where you almost feel as if the people who are saying that there's no risk in these technologies almost don't really believe that they're that powerful. Yeah. They, they, they almost don't believe that there could be loads of good things as well because you have, if you accept that there are good things there, there have to be bad effects with these models too. Right. Yeah. So you're almost like denying the, the power of these models.
There's a kind of a, a a weird dynamic where you almost feel as if the people who are saying that there's no risk in these technologies almost don't really believe that they're that powerful. Yeah. They, they, they almost don't believe that there could be loads of good things as well because you have, if you accept that there are good things there, there have to be bad effects with these models too. Right. Yeah. So you're almost like denying the, the power of these models.
**Stuart Ritchie：** 我最近听到的最精辟的说法是：今天的"加速主义者"（accelerationist）实际上是技术悲观主义者。
**Stuart Ritchie:** The, the best take I heard on this recently was someone who noted that today's like accelerationist are actually technological pessimists. Right?
**Jack Clark：** 对。
因为他们认为技术只会从现在的基础上再前进一小步然后就停了。如果你是一个真正的加速主义者，你会对这些东西持续变强的后果感到震撼和敬畏，以及一丝恐惧。我还跟政策制定者讲另一个道理：看，如果我们错了——如果正如你在对话开头说的那样，这项技术碰了壁，我们今天就停下来——很好。我们将获得大量好处和可能很少的风险，我们能应对。但如果我们对了，我们将需要全新的制度、新的政府体系，需要同时面对巨大的富足和巨大的威胁。所以我想……希望我们是对的。或者希望我们是错的。这部分仍然是最不清晰的地方之一。
是的，确实。让我们回到政策问题。在未来一年，有各种峰会要召开。各国政府以不同方式应对 AI——有些更担心安全问题，有些更相信经济效益。你密切关注政策领域。你觉得接下来一年左右会发生什么？
这将是非常非常忙碌的一年。也别忘了，我们和其他实验室在接下来一年里会产出更好的 AI 系统。这几乎是默认假设了，但值得明确说出来——系统会变得更好。安全峰会将继续：2023年是 Bletchley Park，今年是首尔，明年2月将是法国峰会。届时各国将聚在一起讨论安全、AI 系统和国际协调。还有美国总统大选这件"小事"——
因为他们认为技术只会从现在的基础上再前进一小步然后就停了。如果你是一个真正的加速主义者，你会对这些东西持续变强的后果感到震撼和敬畏，以及一丝恐惧。我还跟政策制定者讲另一个道理：看，如果我们错了——如果正如你在对话开头说的那样，这项技术碰了壁，我们今天就停下来——很好。我们将获得大量好处和可能很少的风险，我们能应对。但如果我们对了，我们将需要全新的制度、新的政府体系，需要同时面对巨大的富足和巨大的威胁。所以我想……希望我们是对的。或者希望我们是错的。这部分仍然是最不清晰的地方之一。
是的，确实。让我们回到政策问题。在未来一年，有各种峰会要召开。各国政府以不同方式应对 AI——有些更担心安全问题，有些更相信经济效益。你密切关注政策领域。你觉得接下来一年左右会发生什么？
这将是非常非常忙碌的一年。也别忘了，我们和其他实验室在接下来一年里会产出更好的 AI 系统。这几乎是默认假设了，但值得明确说出来——系统会变得更好。安全峰会将继续：2023年是 Bletchley Park，今年是首尔，明年2月将是法国峰会。届时各国将聚在一起讨论安全、AI 系统和国际协调。还有美国总统大选这件"小事"——
**Jack Clark:** Right.
Yeah. Because they think it just accelerates a little further from where it is today and then stops. I think if you are a true accelerationist, you kind of reckon with shock and awe and some small amount of dread of the implications of what happens if this stuff keeps, get betting keeps sort of getting better and better. And another story I sort of tell policy makers is like, look, if we're wrong, and as you said at the start of a conversation, if this technology kind of hits a wall and we just stop it today, great. We're gonna get loads and loads of benefits and some probably small amount of risk and, and we'll manage if we're right, we're going to need kind of new institutions, new systems of government, and we're going to need to reckon with both vast abundance and the potential of vast fret. So I guess let's hope that we're right. Yeah. Or hope that we're wrong. That part still feels like the, one of the, one of the kind of least clear aspects of this.
Yeah, absolutely. Let's return to a policy question in the upcoming year, we've got various summits coming up. Governments are kind of grappling with AI in different ways. Some are more worried than others about the safety stuff. Some are more convinced that this could be an economic benefit and so on. You're, you know, very in touch with the, the, the, the policy world. What do you think is coming up in the next year or so?
So it's going to be a, a really busy, busy year. And also let's not forget that us and the other labs are gonna produce better AI systems during the coming year. It's kind of assumed, but it's worth stating totally that the systems will get better. We have the continuation of the safety summits. There was Bletchley Park in 2023, then there was Seoul in, in Korea this year. There will be the French summit coming up in February next year. That's where countries are gonna convene to think about safety and AI systems and coordination about them. There is the small matter of the presidential election in the US
Yeah. Because they think it just accelerates a little further from where it is today and then stops. I think if you are a true accelerationist, you kind of reckon with shock and awe and some small amount of dread of the implications of what happens if this stuff keeps, get betting keeps sort of getting better and better. And another story I sort of tell policy makers is like, look, if we're wrong, and as you said at the start of a conversation, if this technology kind of hits a wall and we just stop it today, great. We're gonna get loads and loads of benefits and some probably small amount of risk and, and we'll manage if we're right, we're going to need kind of new institutions, new systems of government, and we're going to need to reckon with both vast abundance and the potential of vast fret. So I guess let's hope that we're right. Yeah. Or hope that we're wrong. That part still feels like the, one of the, one of the kind of least clear aspects of this.
Yeah, absolutely. Let's return to a policy question in the upcoming year, we've got various summits coming up. Governments are kind of grappling with AI in different ways. Some are more worried than others about the safety stuff. Some are more convinced that this could be an economic benefit and so on. You're, you know, very in touch with the, the, the, the policy world. What do you think is coming up in the next year or so?
So it's going to be a, a really busy, busy year. And also let's not forget that us and the other labs are gonna produce better AI systems during the coming year. It's kind of assumed, but it's worth stating totally that the systems will get better. We have the continuation of the safety summits. There was Bletchley Park in 2023, then there was Seoul in, in Korea this year. There will be the French summit coming up in February next year. That's where countries are gonna convene to think about safety and AI systems and coordination about them. There is the small matter of the presidential election in the US
**Stuart Ritchie：** 对。是的。
**Stuart Ritchie:** Right? Yep. Yep.
**Jack Clark：** 所以我们将会——
**Jack Clark:** So we're gonna-
**Stuart Ritchie：** 政策大事。是的。
**Stuart Ritchie:** Policy concern. Yeah. I guess
**Jack Clark：** 对，两位候选人之间那"微小的"政策差异。
**Jack Clark:** Yeah. The small policy differences between the two candidates currently.
**Stuart Ritchie：** 是。
**Stuart Ritchie:** Yep, yep.
**Jack Clark：** 其结果将是一届新政府。每届新政府都想在头100天内做出成绩。所以明年1月、2月、3月，不管哪一方执政，我们都可以预期他们会在 AI 方面采取一些可能影响深远的举措。最后，欧盟的 AI 法案即将生效并进入实施阶段。这意味着包括 Anthropic 在内的 AI 公司，到明年这个时候将实际受到欧洲一定程度的监管。作为其中的一部分，欧盟和 AI 办公室以及欧洲委员会必须弄清楚测试、评估等具体意味着什么。
**Jack Clark:** And what's gonna, what that's going to lead to is a new administration. Every administration thinks about getting stuff done in the first 100 days. So January, February, March, we can expect whichever administration is in place to make some moves on AI, which could be quite impactful. And finally, we have the European Union has the AI Act, which is coming into force and is gonna go into kind of implementation mode next year. So that means AI companies, including Anthropic will by this time next year, have actually fallen under some degree of like, regulation in, in Europe. And as part of that, the EU and the AI office and the European Commission is gonna have to figure out what kind of testing and evaluation and everything else means
**Stuart Ritchie：** 是的。想必也会与各国政府及其安全研究所合作。
**Stuart Ritchie:** And, and, and yeah. Presumably working with governments, with their safety institutes and so on.
**Jack Clark：** 还有几件事。我们谈了这里的 AI Safety Institute，但加拿大也在成立一个，日本也在成立一个，还有很多其他国家也在成立。我甚至不确定能说出全部名字，因为有些是非公开的、只有我知道。
**Jack Clark:** And, and also just let me just unspool a few other things. You know, there's the, the AI Safety Institute we've spoken about here. Yeah. But there's one being stood up in Canada. There's one being stood up in Japan. There's one being stood up in like many other countries. I'm not even sure I can name all of them because some of them are private and known only to me.
**Stuart Ritchie：** 对。
**Stuart Ritchie:** Right.
**Jack Clark：** 是的。有很多。
**Jack Clark:** Yeah. But there's lots of them.
**Stuart Ritchie：** 对。
**Stuart Ritchie:** Right,
**Jack Clark：** 所以将会出现一个政府"大使馆"网络——面向这个新的硅基民族国家——在全球各地建立起来。另外中国也越来越多地在联合国提出 AI 议题，因为中国觉得自己可能没有充分融入国际对话，所以试图利用联合国作为平台来讨论 AI。我们这周录制时联合国大会正在召开。我们和国务院宣布将以补贴方式向全球提供 Claude 的使用。很多其他公司也做了类似的事情。国际上正在发生大量行动。如果有人以为 AI 在减速，各国政府现在可是全都醒了。明年，2025年，我们会看到政策领域发生疯狂的新变化。所以我会很忙。
确实。当你跟这些政策制定者交流时——不管是在工党大会、美国还是其他国家——有没有一件事是你每次都会特别强调的？你在运用各种隐喻、解释这些系统如何工作。有没有一个核心信息是你试图传达给每一个人，来真正抓住他们对这个问题的注意力？
确实。当你跟这些政策制定者交流时——不管是在工党大会、美国还是其他国家——有没有一件事是你每次都会特别强调的？你在运用各种隐喻、解释这些系统如何工作。有没有一个核心信息是你试图传达给每一个人，来真正抓住他们对这个问题的注意力？
**Jack Clark:** Right. So there's gonna be this network of government embassies to this new silicon nation state that's being built, being stood up around the world. And also China is increasingly raising the issue of AI at the UN because China feels that it's probably not as integrated into the international conversation as it could be. And so it's trying to use the UN as a venue to raise AI. And obviously we have the UN General Assembly happening this week while we're talking, we announced there the State Department that we're going to make Claude available to kind of people around the world in a, in a subsidized way. Many other companies did the same. There's just huge action happening internationally. And I think if people thought AI is kind of slowing down, the governments are all awake now. So next year, 2025, we're gonna see wild new, new things happening in policy. So I'm gonna be busy.
Yes, indeed. Indeed. So when you're talking to these policy makers, either at the Labor Party conference or in the US or in other countries, is there one thing that you tend to, to, to say, I mean, you're deploying the various metaphors and you're talking about how these systems work. What's the kind of one thing that you try and get across to, to everyone to really grab their attention on this issue?
Yes, indeed. Indeed. So when you're talking to these policy makers, either at the Labor Party conference or in the US or in other countries, is there one thing that you tend to, to, to say, I mean, you're deploying the various metaphors and you're talking about how these systems work. What's the kind of one thing that you try and get across to, to everyone to really grab their attention on this issue?
**Stuart Ritchie：** 我总是会说：你知道这些公司的领导者在说什么吧——Dario、Sam、Demis 在谈通用人工智能（AGI）的时候，那不是营销术语，是他们真心相信的东西。他们相信自己有机会构建一种通用的合成智能，具有人类的创造力——如果运行在机器速度上的话。如果我们中任何一家成功了，那将是完全改变世界的事。我就是想让他们记住：是的，我们今天有各种现实问题要处理——怎么测试这些系统、怎么融入经济。但如果我们是对的，真正疯狂的事情将会发生，将需要比任何普通技术都大得多的政策响应规模。我只是想让他们明白，这是实验室里的人真正相信的事情。
**Stuart Ritchie:** I, I try to, I always say, you know what the, the leaders of these companies are saying what, you know, Dario and Sam and Demis are all saying when they talk about artificial general intelligence, it's not a marketing term, it's a general thing that they believe in. They believe that they are, have a chance of building a generally intelligent kind of synthetic intelligence with the, the creativity of a human, you know, if it runs at machine speed. And that is a completely world changing thing if any of us succeed. And I just try and leave them with like, yes, we have all of these problems around us today. We have to obviously think about how we test these systems, how we integrate from into the economy, but if we're right, really, really wild stuff is going to happen, but we'll demand a much larger scale policy response than any kind of normal technology has in the past. And I just try and impress upon them that that's a real thing that these people at these labs believe.
**Jack Clark：** 是的，我们所处的局面不是正常的。
**Jack Clark:** Yeah, this is not a normal situation that we're in.
**Stuart Ritchie：** 基本上，无论是巨大经济丰裕的上行方面，还是某些滥用或风险的下行方面，都不会是正常的。它们的影响规模将是异常巨大的。值得确保我们理解这一点。
**Stuart Ritchie:** Basically neither the, the, the upsides of the, the immense kind of economic abundance nor potentially the downsides like certain misuses or risks, neither of these will be normal. They'll be like abnormally sized in their effect and it's worth leading sort of making sure we understand that.
**Jack Clark：** Jack，除了订阅你的 newsletter、关注 Anthropic 网站的研究动态之外，人们应该做什么来跟上这些？我注意到一点——很多人都注意到了——AI 方面的信息量简直无穷无尽。你没法关注世上其他任何事情，因为每天都有那么多 AI 新闻：新模型、新应用、新的奇怪事物。你自己怎么跟上这些然后为人们做总结的？
一个方法就是使用 AI 系统本身。很早期的 Claude，我会跟它聊天，觉得有点用，但更多是一种新奇玩物。我想"嗯，我们在做的这个 AI 东西挺有趣的，能让它做一些好玩的事"。但现在我真的在用它——像工具一样用。一部分是因为我通过大量对话学会了怎么从它身上获得最大价值，另一部分是因为技术本身变得更强大、更实用了。所以我觉得人们应该试用现在可用的 AI 工具。包括 Claude 在内的很多工具都是免费的，直接用就行。我还建议人们试着去想象一下：在乐观的情境下，对一个人来说"能做什么"的唯一真正限制，可能将变成你自己的创造力以及你如何运用它。我和你都有小孩子，我一直在想他们将来上学的事。我觉得主要是：用这些工具，但最重要的是要保持极度的创造力。因为创造力才能让你从中获得最大价值，找到所有出人意料的用法。
这往往是人们遇到的困难——他们在手机上下载了 Claude 或者打开 claude.ai，然后坐在那里想"好吧，我现在该干什么？"正如你说的，限制在于你需要想出酷的事情来做。它能做的太多了。我们可以指引人们一些方向，我们有很多想法给大家。但最惊人的用途将来自那些极具创造力的人类想出的酷东西。
一个方法就是使用 AI 系统本身。很早期的 Claude，我会跟它聊天，觉得有点用，但更多是一种新奇玩物。我想"嗯，我们在做的这个 AI 东西挺有趣的，能让它做一些好玩的事"。但现在我真的在用它——像工具一样用。一部分是因为我通过大量对话学会了怎么从它身上获得最大价值，另一部分是因为技术本身变得更强大、更实用了。所以我觉得人们应该试用现在可用的 AI 工具。包括 Claude 在内的很多工具都是免费的，直接用就行。我还建议人们试着去想象一下：在乐观的情境下，对一个人来说"能做什么"的唯一真正限制，可能将变成你自己的创造力以及你如何运用它。我和你都有小孩子，我一直在想他们将来上学的事。我觉得主要是：用这些工具，但最重要的是要保持极度的创造力。因为创造力才能让你从中获得最大价值，找到所有出人意料的用法。
这往往是人们遇到的困难——他们在手机上下载了 Claude 或者打开 claude.ai，然后坐在那里想"好吧，我现在该干什么？"正如你说的，限制在于你需要想出酷的事情来做。它能做的太多了。我们可以指引人们一些方向，我们有很多想法给大家。但最惊人的用途将来自那些极具创造力的人类想出的酷东西。
**Jack Clark:** Jack, aside from signing up to your newsletter and, you know, keeping eye on the Anthropic site for research updates and so on, what, what do, what should people do to keep up with this? I mean, one of the, one of the things that I certainly notice that lots people notice is that it's just an endless amount of stuff on AI. Like, you can't, you can't focus on any other things in the world. Yeah. Because there's just so much AI news happening every day. New models, new uses, new weird things coming up. What, what, how do you go about trying to keep up with this stuff and then summarizing it for people?
One thing is just using the AI systems themselves and very early versions of Claude, I would sort of talk to and find vaguely useful, but also kind of like a curiosity. And I was like, huh, this AI stuff we're working on, it's kind of interesting. I can get it to do funny things. But now I actually just use it, I use it like a tool partly because I've learned how to get most outta it by talking to it a lot. But also that the technology has just become a lot more capable and a lot more useful. And so I think that people should try to use the AI tools that are available today. Many of them, including Claude, are available for free. You can just start using it. I would also recommend that people, I guess just try and reckon with the, the implications of a world where the only real limiter on what you can do as a person is probably going to become under optimistic versions of this scenario. Your own kind of creativity and how you use that. I mean, you and I both have, both have young kids and I've been thinking about this since we're gonna go to school at some point. Mostly I think, oh, you should, you should like use these tools, but you should mostly just like be incredibly creative. Because it's the creativity stuff that is going to allow you to kind of get the most out of it and, and use this in all the unexpected ways.
Well, that's often the, the, the trouble people have is they, they they get, you know, the Claude app on their phone or whatever. Yeah. They go, they go to claude.ai and then they sit and go, okay, what, what do I do now? And the, as you say that the limit is, is you need to think of cool things to do. Yeah. You know, this can do so much stuff. And that I think, you know, we can, we can help by pointing people in the right direction. Yeah. We have lots of ideas for people and so on. But yeah, the real, the real, you know, amazing uses are gonna come from just incredibly creative humans thinking of cool things to do with these.
One thing is just using the AI systems themselves and very early versions of Claude, I would sort of talk to and find vaguely useful, but also kind of like a curiosity. And I was like, huh, this AI stuff we're working on, it's kind of interesting. I can get it to do funny things. But now I actually just use it, I use it like a tool partly because I've learned how to get most outta it by talking to it a lot. But also that the technology has just become a lot more capable and a lot more useful. And so I think that people should try to use the AI tools that are available today. Many of them, including Claude, are available for free. You can just start using it. I would also recommend that people, I guess just try and reckon with the, the implications of a world where the only real limiter on what you can do as a person is probably going to become under optimistic versions of this scenario. Your own kind of creativity and how you use that. I mean, you and I both have, both have young kids and I've been thinking about this since we're gonna go to school at some point. Mostly I think, oh, you should, you should like use these tools, but you should mostly just like be incredibly creative. Because it's the creativity stuff that is going to allow you to kind of get the most out of it and, and use this in all the unexpected ways.
Well, that's often the, the, the trouble people have is they, they they get, you know, the Claude app on their phone or whatever. Yeah. They go, they go to claude.ai and then they sit and go, okay, what, what do I do now? And the, as you say that the limit is, is you need to think of cool things to do. Yeah. You know, this can do so much stuff. And that I think, you know, we can, we can help by pointing people in the right direction. Yeah. We have lots of ideas for people and so on. But yeah, the real, the real, you know, amazing uses are gonna come from just incredibly creative humans thinking of cool things to do with these.
**Stuart Ritchie：** 还有一个简单的主意：花将近10年时间写350个短篇故事，然后问 Claude 对这些故事有什么看法。今天就可以开始了。
**Stuart Ritchie:** And one straightforward idea is just spend almost 10 years writing 350 short stories and then ask Claude what it thinks of them. That's like an easy thing you can get started with today.
**Jack Clark：** 这就是你开始使用 AI 的方式。没错。谁知道10年后世界会是什么样子呢。Jack，非常感谢你今天和我对话。太感谢了，非常愉快。大家可以在 anthropic.com 了解更多，也可以去 Jack 的 Substack——叫什么来着？importai.substack.com。对，就是这个。非常感谢收看，下期见。
**Jack Clark:** There you go. There's your way to start with AI. Exactly. I mean, who knows what the world might look like in 10 years after this. Jack, thank you so much for, for talking to me today. Oh, thanks very much. It's been a great pleasure. And you can find out more on anthropic.com, but also on Jack's Substack, which is what's the- - importai.substack.com. That's the one. Thank you so much for watching and I'll see you in the next one.
