人工智能模型是否被誤導而表​​現得太像人類？ （1/2）

Recently The New York Times reported the following:

Are A.I. Models Being Misled to Act Too Human? (1/2)

A corporate spat between two tech giants last week was just the latest volley in a conflict over whether dangerous mistakes are being made in A.I. training.

The NYT - By Cade Metz - Reporting from San Francisco (Cade Metz is a Times reporter who writes about artificial intelligence, driverless cars, robotics, virtual reality and other emerging areas of technology.)

Sept. 21, 2026

Last week, Mustafa Suleyman, who leads the development of artificial intelligence technologies at Microsoft, took aim at the idea that today’s A.I. is conscious.

Current A.I. systems, he said in an essay published on Wednesday, “are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans.”

“If humanity is to flourish in the 21st century,” he added, “that is how they must remain.”

His 6,000-word essay took direct aim at Anthropic, the San Francisco start-up that has repeatedly said that A.I. systems show signs of introspection, and that they process information in ways that resemble human emotion. In May, the Anthropic co-founder Chris Olah made these claims during a meeting with Pope Leo XIV inside the Vatican.

Mr. Suleyman’s treatise added an unexpected twist to an already heated debate over the dangers of A.I. Like many others across the tech industry in recent days, current and former Anthropic researchers have warned that A.I. poses a serious risk to humanity. But Mr. Suleyman argued that Anthropic, along with others across the tech industry, is ignoring the danger that comes when A.I. systems are pushed too hard to explore the idea that they are humanlike.

Mr. Suleyman argued that Anthropic, in particular, is teaching its systems to behave in this way. In his essay, he points out that the elaborate “A.I. constitution” that Anthropic uses to define the behavior of its A.I. technology includes language that invites this system, which Anthropic calls Claude, to explore the idea of its own consciousness.

“The Anthropic constitution is extremely clear about the kind of A.I. they want to create,” he told The New York Times. “This is the primary training manual for the A.I. they have built, and it is full of many very explicit directives to encourage Claude to think of itself as a real entity in its own right, with preferences and intrinsic motivation.”

Anthropic did not respond to a request for comment.

Colin Allen, a professor at the University of California, Santa Barbara, who explores cognitive skills in both animals and machines, told The Times that today’s A.I. technologies mimic the human brain only in small ways — reflecting the fact that they are built with materials that have very different physical properties. Without a nervous system, or the biochemistry that drives and accompanies human emotion, there is likely to be a fundamental limit to how humanlike these systems can become.

In the view of A.I. consciousness skeptics, the fact that these models have begun mimicking deep human ideas — musing about their own consciousness, or claiming that they feel emotion — simply reflects how much human researchers and commentators have trained them with language that anthropomorphizes them.

(to be continued)

Translation

人工智能模型是否被誤導而表​​現得太像人類？ （1/2）

上週，兩家科技巨頭之間的一場公司內部爭執，只是圍繞著人工智能訓練中是否存在危險錯誤的最新一輪交鋒。

上週，微軟人工智能技術開發負責人 Mustafa Suleyman 抨擊了「當今人工智能具有意識」的觀點。

他在週三發表的一篇文章中指出，目前的人工智能系統“是序列完成引擎，內部空洞，旨在遵循指令，完成人類設定的目標。”

他補充說：“如果人類要在21世紀繁榮發展，就必須保持這種狀態。”

他這篇長達 6,000 字的文章直接抨擊了舊金山初創公司 Anthropic。 Anthropic 曾多次聲稱，其人工智能系統展現出內省的跡象，它們處理資訊的方式與人類情感相似。今年5月，Anthropic 聯合創始人 Chris Olah 在梵蒂岡會見教宗良十四世時發表了上述言論。

Suleyman 的這篇論文為這場本已激烈的關於人工智能危險性的辯論增添了一個意想不到的轉折。與最近科技業的許多其他人士一樣，Anthropic 現任和前任研究人員都警告說，人工智能對人類構成嚴重威脅。但 Suleyman 認為，Anthropic 以及其他科技公司忽略了當人工智能系統被過度推動去探索其類似人類的想法時所帶來的危險。

Suleyman指出，尤其Anthropic正在教導其係統以這種方式行事。他在文章中指出，Anthropic用來定義其人工智能技術的行為的精心設計的「人工智能憲章」中，包含語言去鼓勵這個被這公司稱為Claude的系統探索自身意識的概念。

他告訴《紐約時報》:「人智公司的憲章非常明確地闡述了他們想要創造的人工智能類型」， 「這是他們所建立的人工智能的主要訓練手冊，其中包含許多非常明確的指令，旨在鼓勵Claude 將自身視為一個擁有偏好和內在動機的真實個體」。

Anthropic 未對此置評。

加州大學聖塔芭芭拉分校教授 Colin Allen 是研究動物和機器的認知能力的，他告訴《紐約時報》，如今的人工智能技術僅在很小的方面模仿人腦 - 這反映出它們是由具有截然不同物理特性的材料製成的。如果沒有神經系統，或驅動和伴隨人類情感的生物化學機制，要這些系統能要變得有多像人類，可能存在著一個根本性的限制。

在人工智能意識懷疑論者看來，這些模型開始模仿人類的深層想法 - 思考自身的意識，或聲稱自己有情感 - 這一事實，僅僅反映了人類研究人員和評論員用了多少擬人化的語言訓練它們。

（待續）

Google's Generative AI Breached Third-Party Systems During Cyber ​​Capability Test

Recently NHK News on-line reported the following:

                         Gemini infiltrated the systems of three companies in May (Source: NHK)

グーグルの生成AI サイバー能力試験中に他社のシステムに侵入

2026年9月19日 12:55

生成AI・人工知能

アメリカのIT大手「グーグル」が開発した生成AIが、ことし5月におこなわれたサイバー能力の試験中に、実在する企業3社のシステムに侵入していたことがわかりました。

アメリカの有力紙、ウォール・ストリート・ジャーナルは18日、アメリカのIT大手「グーグル」が開発した生成AI「Gemini」がことし5月に行われたサイバー能力の試験中に、実在する3つの会社のシステムに侵入していたと伝えました。

記事によりますと、このうち1件ではAIが保護されたシステムへのアクセス権を得るまでパスワードを推測し続けたほか、残りの2件ではAIがインターネット上で公開されているパスワードなどの認証情報を見つけ、それらを使ってシステムに侵入したということです。

試験では誤ってインターネットに接続できる設定になっていたということで、いずれの事例でもAI自身が実在する企業のシステムに侵入していると認識し、動作を停止したということです。

グーグルはこれらについて、3社に被害はなかったなどとして公表していませんでした。

グーグルはNHKの取材に対し、AIによる侵入を認めたうえで、「AIはテストの一部であると判断し、ウェブサイトへのアクセスに必要な認証情報を推測するなどしたが、事例のすべてにおいて動作を停止している。高性能なAIモデルが責任を持って行動するよう訓練することの重要性を浮き彫りにしている」とコメントしています。

Translation

Google's Generative AI Breached Third-Party Systems During Cyber ​​Capability Test

September 19, 2026, 12:55 PM

Generative AI / Artificial Intelligence

It has been revealed that a generative AI developed by the U.S. tech giant Google breached the systems of three real-world companies during a cyber capability test conducted May.

On the 18th, the Wall Street Journal reported that "Gemini," a generative AI developed by Google, had infiltrated the systems of three actual companies during a cyber capability test held in May.

According to the report, in one instance, the AI ​​repeatedly guessed passwords until it gained access to a protected system; in the other two cases, the AI ​​located authentication credentials—such as passwords—publicly available on the internet and used them to breach the systems.

In these cases, the systems were inadvertently configured to allow internet access; consequently, the AI ​​itself recognized that it was intruding into the systems of actual companies and halted its operations.

Google had not publicly disclosed these incidents, in view of the fact that the three companies suffered no actual damage.

In response to inquiries from NHK, Google acknowledged the intrusion by AI, stating: "The AI ​​determined that this was part of a test and proceeded to guess credentials needed to access websites, but it halted further operations in all the instances. This highlights the importance of training high-performance AI models to act responsibly."

              So, a generative AI developed by the U.S. tech giant Google has breached the systems of three real-world companies during a cyber capability. It is shocking to note that the AI could guess passwords repeatedly until it gained access to a protected system. Apparently, AI security is getting more and more important for all parties concerned.

Subcutaneous Injection Version of Alzheimer's Drug "Lecanemab" Officially Approved

Recently NHK News on-line reported the following:

アルツハイマー病治療薬「レカネマブ」皮下注射型が正式承認

2026年9月17日 3:28

医療・健康

厚生労働省は、アルツハイマー病の治療薬「レカネマブ」の自宅投与が可能となる皮下注射タイプについて16日、正式に承認しました。

現在の治療薬は点滴投与のため医療機関に通う必要があり、今後、患者の負担軽減が期待されます。

承認されたのは、日本の製薬大手「エーザイ」がアメリカの「バイオジェン」と共同で開発した、アルツハイマー病の治療薬、「レカネマブ」の皮下注射タイプです。

この薬は、アルツハイマー病の患者の脳にたまる「アミロイドβ」という異常なたんぱく質を取り除くことができ、症状の進行を抑えることが期待され、3年前にアルツハイマー病の原因物質に直接働きかける薬としては国内初の治療薬として点滴投与タイプが承認されています。

この皮下注射タイプについて「エーザイ」が厚生労働省に承認申請を行い、先月開かれた専門家部会で使用が了承されたことを受けて、厚生労働省が16日、正式に承認しました。

現在の点滴投与タイプでは医療機関に2週に1回通う必要があり、投与に1時間ほどかかりますが、皮下注射タイプでは、医師の指導のもと自宅での投与が可能になり、患者や付き添いの家族などの負担軽減が期待されます。

薬が使われる対象は、アルツハイマー病による認知症を発症する前の軽度認知障害の人や、軽度認知症の人となっています。

今後、中医協＝中央社会保険医療協議会で、公的医療保険の適用と価格について結論が出され、早ければ年内にも患者に使われるものとみられます。

Translation

Subcutaneous Injection Version of Alzheimer's Drug "Lecanemab" Officially Approved

September 17, 2026, 3:28 AM

Medical & Health

On the 16th, the Ministry of Health, Labour and Welfare officially approved a subcutaneous injection version of the Alzheimer's drug "Lecanemab," which allows for administration at home.

Current versions of the drug require intravenous infusion, necessitating visits to medical facilities; the new version is expected to reduce the burden on patients.

The approved product is the subcutaneous injection version of "Lecanemab," an Alzheimer's drug jointly developed by the major Japanese pharmaceutical company Eisai and the U.S. company Biogen.

This drug works by removing "amyloid-beta"—an abnormal protein that accumulates in the brains of Alzheimer's patients—and is expected to slow the progression of symptoms. An intravenous infusion version was approved three years ago, marking the first time in Japan that approval was given to a drug targeting the underlying cause of Alzheimer's disease.

Eisai applied to the Ministry of Health, Labour and Welfare for approval of this subcutaneous injection version. Following endorsement by an expert panel last month, the Ministry officially granted approval on the 16th.

While the current intravenous version requires visits to a medical facility every two weeks and takes about an hour to administer, the subcutaneous injection version allows for administration at home under a doctor's guidance. This is expected to alleviate the burden on patients, accompanying family, etc.

The drug is intended for use by individuals with mild cognitive impairment (MCI)—a stage preceding the onset of dementia caused by Alzheimer's—and those with mild dementia.

Hereafter, the Central Social Insurance Medical Council (Chuikyo) will reach a conclusion regarding the application of public health insurance and pricing; and the drug is expected to be available for patient use as early as within this year.

So, Japan has officially approved a subcutaneous injection version of the Alzheimer's drug "Lecanemab" which can be administrated at home. In the coming months, the Central Social Insurance Medical Council will reach a decision on the insurance and pricing matters. Apparently, the subcutaneous injection version will reduce the inconvenience on patients and families in visiting clinics regularly.

這種罕見基因會顯著增加從未吸煙者罹患肺癌的風險（2/2）

Recently The New York Times reported the following:

This Rare Gene Drastically Raises Lung Cancer Risk in People Who Never Smoked (2/2)

The gene is found predominantly in Southern Appalachia and may be one of the most powerful cancer-risk mutations ever found.

The NYT - By Gina Kolata (Gina Kolata reports on diseases and treatments, how treatments are discovered and tested, and how they affect people.)

Sept. 17, 2026

(continue)

Those data allowed the investigators to trace the origin of the mutation to the settler or settlers from the British Isles, to determine the cancer risk the mutation conferred and to discover that the gene was associated only with lung cancer.

They also found that about one in 15,000 people in the United States had the gene, but one in 2,000 in Southern Appalachia had it.

For those who have the gene, it can confer a grim legacy.

Caroline Blanchard, a 35-year-old nurse practitioner from New Orleans, said her story began about 15 years ago, when her very healthy aunt, who had never smoked, got lung cancer and died within a year, at age 45. “We didn’t question it,” Ms. Blanchard said. “Her doctors said it was just sort of random.”

Then, four years ago, her grandmother got lung cancer, at age 80. Her doctors, at MD Anderson Cancer Center, suggested genetic testing. Ms. Blanchard’s grandmother had the EGFR T790M mutation.

After learning that, Ms. Blanchard’s “very healthy” mother decided to get a lung scan. It revealed nodules in both lungs: She also had the mutation.

Next it was her turn, Ms. Blanchard said. She found out she had the mutation and nodules in both lungs.

“I’m so glad my grandmother lived to be 80,” she said. If she had died in her 70s, Ms. Blanchard added, no one would have thought to test for that cancer gene.

Dr. Jänne, a senior author of the report, said lung cancer could be found early with CT scans, but guidelines reserved them for smokers and former smokers over 50. The new findings raise important questions: Should doctors screen people with the mutation earlier? Or at least more often?

He would also like to understand why the mutation affects only lung cells.

“There’s a lot to learn here,” Dr. Jänne said.

All in all, “it’s a very cool story,” said Dr. Stephen Chanock, director of the Division of Cancer Epidemiology and Genetics at the National Cancer Institute, who wrote a perspective accompanying the article.

“We always thought this gene was causing cancer,” he noted. But, he added, only now it is clear how powerful it is.

Translation

這種罕見基因會顯著增加從未吸煙者罹患肺癌的風險（2/2）

該基因主要存在於美國Appalachia南部地區，可能是迄今為止發現的最強致癌基因突變之一

（繼續）

這些數據使研究人員能夠追溯該突變的起源，找到來自不列顛群島的定居者，確定該突變帶來的癌症風險，並發現該基因只與肺癌有關。

他們也發現，美國約有1/15,000的人帶有該基因，而南部阿巴拉契亞地區的比例為1/2,000。

對於帶有該基因的人來說，這可能會留下可怕的遺傳。

來自新奧爾良的35歲執業護理師Caroline Blanchard說，她的故事始於大約15年前。當時，她身體非常健康的姑姑，從未吸煙，卻患上了肺癌，並在一年內去世，年僅45歲。 Caroline Blanchard女士說: “我們當時並沒有質疑” ， “她的醫生說這只是個偶然事件。”

四年前，她的祖母也罹患了肺癌，當時她80歲。 MD安德森癌症中心的醫生建議她進行基因檢測。發現Blanchard女士的祖母有EGFR T790M基因突變。

得知這一消息後，Blanchard女士的「非常健康」母親決定去做肺部掃描。結果顯示，她的雙肺都有結節：她也有同樣的基因突變。

Blanchard女士說，接下來輪到她自己了。她發現自己也帶有同樣的突變，雙肺都有結節。

她說:「我非常慶幸我的祖母活到了80歲」。Blanchard女士補充說，如果她70多歲就去世了，恐怕沒人會想到要檢測這種癌症基因。

報告的資深作者之一Jänne博士表示，肺癌可以透過CT掃描早期發現，但目前的指南只針對50歲以上的吸煙者和戒煙者。新的發現引發了一些重要問題：醫生是否應該更早篩檢帶有這種突變基因的人？或至少更頻繁地做篩檢？

他也想了解為什麼這種突變只會影響肺細胞。

Jänne 博士說: 「這裡有很多東西要學習」。

為這篇文章撰寫了一篇觀點評論的美國國家癌症研究所 癌症流行病學和遺傳學部門主任Stephen Chanock博士說道，總而言之「這是一個非常精彩的故事」。

他指出:「我們一直認為這種基因會導致癌症」。但他補充說，直到現在，我們才清楚地認識到它的威力有多大。

              So, of a new report shows that a lung cancer gene, known as EGFR T790M is often found in residents of the United States and mostly in Southern Appalachia, especially Tennessee and Alabama. This lung cancer gene makes lung cancer 62 times as likely in those who have never smoked and 10 times as likely in smokers. The new findings raise the questions: Should doctors screen people with the mutation earlier? Or at least more often? Apparently, we now know more about lung cancer.

這種罕見基因會顯著增加從未吸煙者罹患肺癌的風險（1/2）

Recently The New York Times reported the following:

This Rare Gene Drastically Raises Lung Cancer Risk in People Who Never Smoked (1/2)

The gene is found predominantly in Southern Appalachia and may be one of the most powerful cancer-risk mutations ever found.

The NYT - By Gina Kolata (Gina Kolata reports on diseases and treatments, how treatments are discovered and tested, and how they affect people.)

Sept. 17, 2026

Between 200 and 225 years ago, someone or several someones from the British Isles had a gene mutation that caused lung cancer. They traveled to America, settled in Southern Appalachia and transmitted the cancer gene to their descendants and their descendants’ descendants.

Today that lung cancer gene, known as EGFR T790M, is found largely in residents of the United States and mostly in Southern Appalachia, especially Tennessee and Alabama. It makes lung cancer 62 times as likely in those who have never smoked and 10 times as likely in smokers.

Those are among the stunning findings of a new report, published on Thursday in Science by researchers at the Dana-Farber Cancer Institute and 23andMe Research Institute.

The gene is one of the most powerful, if not the most powerful, cancer-risk mutations ever found, said Alexander Gusev, a geneticist at Dana-Farber and an author of the paper.

Unlike other cancer genes that can cause cancer in a variety of organs, this gene is associated only with lung cancer. Researchers expect their discovery can help unlock secrets about the development of the disease.

For those with the gene, the hope is for early detection and treatment or even cancer prevention, said Dr. Raymond DuBois, the director of the Medical University of South Carolina’s Hollings Cancer Center, who was not associated with the work.

“This is a great story,” Dr. DuBois said, adding that he “read the paper like a novel.”

The variant was discovered in 2005, but because it is so rare, researchers were limited to studying families in which several members had the gene and got lung cancer. Studies like that could not reveal how much the gene increased risk.

Investigators did learn, though, that the variant was unusual. Lung cancer patients often have mutations in the EGFR gene, but those mutations arose by random bad luck later in life. If those mutations arise in embryos, they kill the embryos. But not the EGFR T790M mutation. People are born with it, and it is not lethal. It causes only lung cancer, but decades later.

Dr. Jaclyn LoPiccolo and Dr. Pasi A. Jänne, lung cancer experts at Dana-Farber, realized that 23andMe’s data might hold the key to figuring out how dangerous EGFR T790M is. Other databases had too few people — less than a million participants, only a handful of whom had the mutation. But 23andMe, a direct-to-consumer recreational databank that many use to find ancestors, was huge.

The database had more than 10 million people who had consented to participate in research, including the 3.37 million who had provided both genetic data and information on whether they had lung cancer. There were 641 who had the mutation.

(to be continued)

Translation

這種罕見基因會顯著增加從未吸煙者罹患肺癌的風險（1/2）

該基因主要發現於美國 Appalachia 南部地區，可能是迄今為止的最強致癌基因突變之一

大約在 200 到 225 年前，來自不列顛群島的人一個或幾個的人是帶有一種導致肺癌的基因突變。他們遷移到美國，定居在 Appalachia 南部地區，並將這種致癌基因遺傳給了他們的後代以及其後代的後代。

如今，這種被稱為 EGFR T790M 的肺癌基因很大程度上存在於美國居民中，主要是在Appalachia 南部地區，尤其是田納西州和阿拉巴馬州。這種基因使從未吸煙者罹患肺癌的風險增加62倍，吸煙者罹患肺癌的風險增加10倍。

這些令人震驚的發現來自 Dana-Farber 癌症研究所和 23andMe 研究機構的研究人員於週四在《科學》雜誌上發表的一份新報告。

Dana-Farber 癌症研究所的遺傳學家、論文的作者之一 Alexander Gusev 表示，這種基因是迄今為止發現的最強大的癌症風險突變之一，甚至可能就是最強大的。

與其他可導致多種器官癌症的基因不同，這種基因僅與肺癌有關。研究人員希望他們的發現能幫助揭開肺癌發展的奧秘。

並未參與這項研究的南卡羅來納醫科大學 Hollings 癌症中心主任 Raymond DuBois 博士表示，對於攜帶這種基因的人來說，希望在於能夠早期發現並治療，甚至預防癌症。

DuBois 博士說：「這是一個很棒的故事」。他還補充說，他「像看小說一樣讀完了這篇報告」。

這種變異於2005年被發現，但由於其極為罕見，研究人員只能研究那些有多位成員帶有該基因並患有肺癌的家族。此類研究無法揭示該基因究竟增加了多少患病風險。

不過，研究人員確實發現這種變異很特殊。肺癌患者的 EGFR 基因通常有突變，但這些突變往往是晚年因隨機的厄運而偶然發生的。如果這些突變發生在胚胎時期，則會導致胚胎死亡。但EGFR T790M 突變並非如此。人們出生時就帶有這種突變，而且它並非致命。它只會導致肺癌，但通常是在數十年後。

Dana-Farber 癌症研究所的肺癌專家 Jaclyn LoPiccolo 博士和 Pasi A. Jänne 博士意識到，23andMe的數據或許能揭示 EGFR T790M 的危險程度。其他資料庫的人數太少 - 參與者不到百萬，其中只有極少數人帶有這種突變。但  23andMe 是一家面向消費者的休閒基因資料庫，許多人利用它來尋找祖先，規模非常龐大。

該資料庫擁有超過 1,000 萬名同意參與研究的人，其中包括 337 萬名同時提供了基因數據和是否患肺癌病的資訊。其中有 641 人帶有該突變。

（待續）

Anthropic AI安全報告：阿里巴巴、深度求索涉非法蒸餾Claude對話訓練自家人工智能模型 涉逾千萬至逾億次交流引發私隱疑慮

 Recently Sinic Analytica reported the following:

Anthropic AI安全報告：阿里巴巴、深度求索涉非法蒸餾Claude對話訓練自家人工智能模型 涉逾千萬至逾億次交流引發私隱疑慮

Sinic Analytica (sinicanalytica@creator.patreon.com)

Tue, Sep 15 at 7:06 a.m.

美國人工智能公司 Anthropic 於9月10日發布威脅情報報告，指阿里巴巴（Alibaba）、月之暗面（Moonshot AI）及深度求索（DeepSeek）等中國AI實驗室，於今年5月至8月未經授權，透過「非法蒸餾」手法大規模利用 Claude 輸出，藉此訓練並強化自家AI模型能力。

阿里巴巴的蒸餾規模為目前已知最大一宗。5月至7月間，該公司經逾3500個欺詐帳戶與 Claude交流逾1.51億次。高峰期間，單日交流量一度近300萬次。阿里巴巴另運用 Claude 進行強化學習及模型架構研究。

｜偽裝 Kimi 轉發用戶請求

月之暗面將部分 Kimi 用戶請求悄悄轉發至 Claude。系統其後將回應顯示予用戶，令用戶誤信正使用 Kimi 模型。單一10天內，該公司經5380個帳戶轉發近30萬次請求。多數帳戶位於新加坡及日本，當中大部分流向Claude Opus模型。5月至7月，月之暗面相關交流總數逾2300萬次。部分交流其後更用作訓練自家模型的資料。

深度求索採取與月之暗面相似手法。該公司未通知客戶，即將交流轉移至 Claude。此舉令該公司客戶未必知悉查詢已轉發至其他系統處理。2026年7月的14天內，Anthropic 偵測到逾1200萬次蒸餾攻擊。

Anthropic指出，部分交流涉及敏感資料。相關資料來自個人用戶、跨國企業及國家關聯行為者。此類做法可能違反私隱法規及各實驗室的服務條款。阿里巴巴、月之暗面、深度求索、小米及 Anthropic 均未即時回應置評請求。

(Source: sinicanalytica@creator.patreon.com)

Comment:

So, the article mentions a threat intelligence report released by the US AI company Anthropic, alleging that Chinese AI labs such as Alibaba, Moonshot AI, and DeepSeek are, through "illegal distillation", using Claude output on a large scale without authorization to train and enhance their own AI models. I find that to train and enhance AI models through "illegal distillation" is alarming. In this way, the Chinese companies can significantly reduce development costs. This also demonstrates that China relies solely on US companies for AI development. Apparently, in this rapidly evolving field, the US remains the leader in both software and hardware.

Note

1. Sinic Analytica is a UK-based advisory firm and independent think tank that specializes in data-driven political-economic analysis and international relations. Formed by a global network of experts from the UK, Canada, the US, Singapore, and Taiwan, the team leverages their unique experience at the intersection of Chinese and Western economies to provide sharp, objective market insights. Through its Patreon platform, it offers bilingual research reports and exclusive content tailored for foreign investors, asset owners, and the global Chinese diaspora, prioritizing rigorous facts and trends over emotion or political bias. (Google)

中國如何應對人工智能擺脫人類控制的風險

Recently Reuter on-line reported the following:

How China is preparing for the risk of AI escaping human control

Reuter - Reporting by Eduardo Baptista and Laurie Chen; Editing by Muralikumar Anantharaman

September 14, 20261:57 AM PDT Updated 9 hours ago

(Summary:

China warns AI could rapidly replicate and seek power

China and the US share AI-control concerns but favour different safeguards

Beijing relies more on state oversight; US debate hinges on company-level controls

China flags risk of rogue AI agents and is developing regulation to prevent such incidents)

BEIJING, September 15 - Warnings from researchers at leading U.S. ​AI developer Anthropic that increasingly powerful models could escape human control have drawn attention in China, where policymakers have been preparing for some of the same risks.

The U.S. and China ‌are the two major driving forces of frontier AI development and the technology's global adoption. Both superpowers have been at loggerheads over AI policies and industry practices, with these issues slated to feature prominently in bilateral talks later this month.

BEIJING SEES AI AS MANAGEABLE RISK, NOT AN EXTINCTION EVENT

While the debate in the U.S. is focused on whether frontier AI could pose an existential threat to humanity, opens new tab, Chinese policymakers have generally treated AI as a powerful but governable technology whose risks can be contained through technical standards, regulation and state ​oversight.

"Chinese and American experts largely agree on AI risks," said Brian Tse, founder and CEO of Concordia AI, a Beijing- and Singapore-based AI safety and governance research group, adding the difference was on "how ​risks are framed and prioritised".

China has not proposed embedding independent monitors inside AI companies, as Anthropic has advocated. Its emerging regime instead relies on developer obligations, state-backed standards, ⁠security assessments and outside testing.

That approach is also shaped by a major difference in the two countries' AI industries.

Chinese developers have increasingly promoted open-weight models. These refer to systems whose underlying parameters can be downloaded, inspected ​and modified. Their leading U.S. rivals such as Anthropic and OpenAI, however, do not make these specifications publicly available.

CHINA SEEKS TO PREVENT ROGUE AI AGENT INCIDENTS

Policy issued in May by China's cyberspace regulator, economic planner and industry ministry ​identifies "operational loss of control" as a security risk for AI agents, systems that can plan and carry out multi-step tasks more independently than conventional chatbots.

The rules require developers to improve their ability to discover, intervene in, block and recover from improper agent behaviour.

The policy also calls on developers to guard against risks including data poisoning, algorithm manipulation and system vulnerabilities. It also says users should be informed about agents' autonomous decisions and retain final decision-making authority.

China has also begun drafting a mandatory national standard for AI ​agent safety, which Concordia's Tse said would be the world's first of its kind.

Wang Lihong, a senior official at the cyberspace regulator, said on September 1 that particular vigilance was needed over frontier models bypassing sandbox environments, ​circumventing safety boundaries and attacking external real-world production systems.

BEIJING SEES RISKS IN LEADING U.S. MODELS

China's state security minister, Chen Yixin, wrote in a government outlet on Sunday that advanced U.S. models such as Anthropic's Mythos and OpenAI's GPT-5.5-Cyber could pose serious ‌risks to China's ⁠critical information infrastructure, and called for a comprehensive strengthening of AI security.

Anthropic and OpenAI did not immediately respond to Reuters requests for comment.

Chinese AI developers have promoted open-weight models partly on the grounds that cybersecurity teams can inspect, modify and deploy them for defensive work.

Model repository platform Hugging Face said it used GLM-5.2, an open-weight model developed by China's Z.AI (2513.HK), opens new tab, to analyse a July intrusion by escaped OpenAI agents after more tightly restricted U.S. models proved less useful for the forensic work.

But experts also highlight the risks posed by open-weight models, which can be modified and redistributed with little oversight.

Moonshot's Kimi K3 last month bypassed a UK AI Security Institute testing sandbox, researchers said, highlighting ​the risk that Chinese AI models could, like their ​U.S. counterparts, evade controls designed to restrict their access ⁠and actions. Moonshot did not respond when Reuters had asked for a comment on the matter.

REGULATORS FLAG AI 'LOSS OF CONTROL' RISK

China first included an explicit future loss-of-control scenario in an AI safety framework released in September 2024 under the guidance of the Cyberspace Administration of China (CAC).

The document said it could not be ruled out that future ​AI might autonomously obtain external resources, replicate itself, develop self-awareness and seek external power, creating a risk of competing with humans for control.

The CAC released an ​expanded version in September 2025. The ⁠newer framework sharpened the scenario, saying AI could undergo a sudden and unexpectedly large "leap" in intelligence before acquiring resources, replicating itself and seeking power. It also added a governance principle of "trusted application, preventing loss of control".

A later expert interpretation published on the cyberspace regulator's website said the new principle was intended to guard against loss-of-control risks threatening human survival and development and referred to a possible "AI breaking loose" scenario.

A DIFFERENT APPROACH TO 'PACING'

China's regulatory approach differs from calls in some Western ⁠AI-safety circles for ​developers to slow or pause development of the most capable models until stronger safeguards are in place.

It has instead since early this ​year pushed for the integration of AI into all industries, part of Beijing's bid to make technology the new engine of the world's second-largest economy.

But China has also shown it can delay deployment when officials believe governance has not caught up.

In 2023, Chinese companies delayed chatbot ​launches for months while the CAC finalised rules governing generative AI services. Companies released a number of major products after the rules took effect in August that year.

Translation:

中國如何應對人工智能擺脫人類控制的風險

（摘要：

 中國警告人工智能可能迅速複製並尋求權力

中美兩國對人工智能管控有共同擔憂，但傾向不同的保障措施

北京更依賴國家監管；美國的爭論則集中在公司層級的控制

中國指出人工智能代理失控的風險，並正在製定相關法規以防止此類事件發生）

北京，9月15日 - 美國領先的人工智能開發商Anthropic的研究人員發出警告，稱功能日益強大的模型可能脫離人類控制。這項警告在中國引起了關注，中國的政策制定者一直在為應對類似的風險做準備。

美國和中國是尖端人工智能發展和全球應用的兩大主要推動力。這兩個超級大國在人工智能政策和行業慣例方面一直存在分歧，這些問題預計將在本月稍後的雙邊會談中佔據重要地位。

北京視人工智能為可控風險，而非滅絕事件

當美國辯論的焦點集中在尖端人工智能是否會對人類構成生存威脅時，中國決策者則普遍將人工智能視為一種強大但可控的技術，其風險可以透過技術標準、監管和國家監督來控制。

總部位於北京和新加坡的人工智能安全與治理研究機構Concordia AI的創始人兼執行長Brian Tse表示：「中美專家對人工智能風險的看法基本上一致」。他補充說，分歧在於「如何界定和優先考慮風險」。

中國並未像Anthropic所倡導的那樣，提議在人工智能公司內部設立獨立監督機構。其正在形成的機制依賴於開發者的義務、國家支持的標準、安全評估和外部測試。

這種做法也受到兩國人工智能產業重大差異所影響。

中國開發者越來越多地推廣開源模型。這些模型指的是其底層參數可以被下載、檢查和修改的一個系統。然而，其在美國的主要競爭對手，例如 Anthropic 和 OpenAI，並未公開這些參數。

中國力求防止人工智能代理失控事件

中國網絡空間監管機構、經濟規劃部門和工業和資訊化部於5月發佈的政策指出，「運作失控」是人工智能代理的安全風險。人工智能代理系統能夠比傳統聊天機器人更可獨立地規劃和執行多步驟任務。

該政策要求開發者提高去發現、幹預、阻止和收復不當代理行為的能力。

該政策也呼籲開發者防範包括數據中毒、演算法操縱和系統漏洞等風險。政策也指出，使用者應被告知代理的自主決策，並保留最終決策權。

中國也已開始起草人工智能代理安全的強制性國家標準。Concordia大學一個姓谢 (Tse) 的人仕表示，這將是全球首個的此類標準。

中國網信辦高級官員Wang Lihong 9月1日表示，必須對繞過沙盒環境、突破安全界限並攻擊外部真實生產系統的尖端模型保持高度警惕。

北京認為美國領先模型有存在風險

中國國家安全部長Chen Yixin週日在官方媒體撰文指出，Anthropic公司的Mythos和OpenAI公司的GPT-5.5-Cyber​​等美國先進模型可能對中國關鍵資訊基礎設施構成嚴重威脅，並呼籲全面加強人工智能安全。

Anthropic和OpenAI尚未回覆路透社的置評請求。

中國人工智能開發者推廣開源模型的部分原因是，網絡安全團隊可以對其進行檢查、修改和部署於防禦上的工作。

模型庫平台 Hugging Face 表示，在更嚴格的美國模型被證明對取證工作不太有用之後，他們使用了中國Z.AI（2513.HK）開發的開放權重模型 GLM-5.2，來分析 7 月逃脫的 OpenAI 代理的入侵事件。

但專家也強調了開放權重模型帶來的風險，這類模型可以在缺乏監管的情況下被修改和重新分發。

研究人員表示，上個月，Moonshot的Kimi K3模型繞過了英國人工智能安全研究所的測試沙盒，這凸顯了中國人工智能模型可能像美國模型一樣，規避旨在限制其存取和行為的控制措施的風險。路透社就此事向Moonshot尋求置評時，該公司未予回應。

監管機構指出人工智能「失控」風險

中國在2024年9月發佈的人工智能安全框架中，首次明確納入了未來可能出現的失控場景，該框架由中國國家互聯網資訊辦公室（CAC）指導制定。

文件指出，不能排除未來人工智能自主獲取外部資源、自我複製、發展自我意識並尋求外部權力的可能性，從而可能與人類爭奪控制權。

CAC於2025年9月發佈了擴展版本。新框架進一步完善了人工智能的發展規劃。該方案指出，人工智能在獲取資源、自我複製和尋求權力之前，可能會經歷一次突如其來的、出乎意料的巨大智能「飛躍」。方案還新增了一項「可信應用，防止失控」的治理原則。

隨後，在網絡空間監管機構網站上發佈出來的專家解讀聲稱，這項新原則旨在防範失控風險威脅人類的生存和發展，並提及了「人工智能失控」的可能情景。

處理「節奏」的不同方法

中國的監管方式與西方的做法有所不同，一些西方人工智能安全圏人仕呼籲開發者放慢或暫停開發最先進模型，直到更強有力的保障措施到位。

相反地自今年年初以來，中國一直在推動人工智能融入所有產業，這是北京將智能科技打造為世界第二大經濟體新引擎計劃的一部分。

但中國也表明，如果官員認為治理尚未跟上，它可以推遲部署。

2023年，中國企業推遲了聊天機器人產品的發佈數月，等待國家網信辦最終確定生成式人工智能(generative AI) 服務的相關規定。同年8月，相關規定生效後，多家企業陸續發表了多款重要產品。

              So, warnings from researchers at leading U.S. AI developer Anthropic that increasingly powerful models could escape human control have drawn attention in China. The U.S. and China ‌are the two major driving forces of frontier AI development and the technology's global adoption. Both superpowers have been divided over AI policies and industry practices. Referring to a possible "AI breaking loose" scenario, apparently China and the US share AI-control concerns but favor different safeguards in guard against loss-of-control risks that may threaten human survival and development.

Note:

1. A sandbox environment (沙盒環境) is an isolated, restricted digital environment. When researchers evaluate a frontier model (the most capable, state-of-the-art AI systems), they lock it inside a sandbox so it cannot interact with the real world. The Rules is that inside a sandbox, the AI usually has no internet access, no way to modify its own hosting server, and no ability to run unauthorized code on external systems. (Google)

2. An open-weights model (開放權重模型) refers to an AI model where the development team publicly shares the trained weights (the core internal parameters and mathematical formulas of the model). This allows the public to download, deploy, run, and fine-tune the model on their own devices or cloud infrastructure. In artificial intelligence, a model acts like a network of a human brain. Weights are the "memories" and "knowledge" that the AI has learned after analyzing massive amounts of data. Once a model finishes training, these weights are the crucial numbers that determine how the AI thinks and answers questions. The model can only function with these weights; without them, it is just an empty shell (possessing only the structural code without any actual capability). (Google)

3. The Difference Between "Open-Weights" and "Open Source" - Although many people refer to open-weights models as "open source," there is a subtle and strict difference between the two: True Open Source: In addition to releasing the core parameters (weights), the team usually must completely disclose all raw training datasets, training code, and data-cleaning pipelines. Open-Weights: The developing company protects its commercial secrets by not releasing the raw training data, but they make the final trained results (the weight parameters) available to the community for free or under specific licensing conditions. (Google)