2026 has heralded the summer of powerful Chinese models: Kimi K3, Qwen3.8, and Z.ai’s GLM-5.3 have arrived in quick succession, all open-weighted and exceedingly close to the closed-model frontier.
At the same time, the AI risk frog pot is heating up globally. China is crucial in enabling humanity to navigate AI risks well, but the Chinese AI safety ecosystem is, in my experience, poorly understood.
In this post, I:
Provide an overview of China’s AI safety ecosystem, which I think is crucial context for thinking seriously about ‘win conditions’ for AI safety;
Highlight concrete ways to contribute to Chinese AI safety, for readers interested in international cooperation.
Content draws on ~3 months working on Sino-Western cooperation and a month-long trip to China in July.
The AI safety landscape
AI risks are receiving increasing recognition in China, including officially: for instance, at WAIC 2026, President Xi Jinping explicitly mentioned loss-of-control risks - a pleasant surprise to many in the Western safety community!
However, the ecosystem still remains small. As a rough estimate, taken from a non-public source: there are likely <100 people in China working full time on frontier AI safety (i.e., catastrophic risk from misuse and LoC), compared to 1,000+ in the West. In addition, annual philanthropic funding for frontier AI safety is around $20 million in China, compared to $1 billion+ in the rest of the world.
Governmental organizations
Regulators
Cyberspace Administration of China (CAC): In May 2026, the CAC and two other agencies issued dedicated guidance on agentic AI safety. As a guiding “opinion,” it has no direct legal force, but it sets a direction that standards bodies can then implement. The CAC also runs the algorithm registry, under which public-facing generative AI services with “public opinion or social mobilization capabilities” must file their algorithms and undergo a security assessment.
Ministry of Industry and Information Technology (MIIT) and Ministry of Public Security (MPS): MIIT and MPS are secondary regulators. MIIT works on industry standards and science-and-technology ethics review rules, and MPS’s Third Research Institute sits on the key standards working group.
Evaluators
CAICT (China Academy of Information and Communications Technology, 中国信通院): a think tank under MIIT, can be thought of as an AISI evaluation arm. Its AI Research Institute runs AI Safety Benchmark, a quarterly evaluation series launched in 2024 with 17-30+ participating institutions.
In February 2026 it launched AI Safety Benchmark 2.0, which includes tests for model deception, loss of control, dangerous-domain misuse, and agentic risks.
In 2025 it evaluated 15 open-source coding models from DeepSeek, Alibaba, and Z.ai for faulty-code and misuse risk (full results).
Shanghai AI Lab: SHLAB functions as a de facto evaluator through the Frontier AI Risk Management Framework (with Concordia AI), which defines seven risk dimensions including biological and chemical risks, and runs frontier model evaluations on biological protocol troubleshooting and hazardous chemical reasoning.
Beijing Institute of AI Safety and Governance (Beijing-AISI): partners with groups like the Chinese Academy of Sciences.
Standards-setting
In China, standards play an important role in operationalizing oft-vague binding regulations.
TC260, the National Information Security Standardization Technical Committee, is the main domestic body. It established a dedicated AI Safety Working Group (WG9) in March 2026, chaired by Zhou Bowen (周伯文), Director of Shanghai AI Lab, with vice-chairs from CNCERT, CAICT, the MPS Third Research Institute, and CNITSEC.
In April 2026, immediately after WG9 was formed, TC260 proposed 16 AI safety standards assigned directly to it.
TC260 also has an AI Safety Governance Framework 2.0 roadmap, which discusses frontier risks.
MIIT/TC1 was established in December 2024, and drafts overlapping standards on agent security, AI-generated content labeling, and ethics review.
AIIA (AI Industry Alliance of China, 中国人工智能产业发展联盟): AIIA is a convening body, founded by CAICT in 2017 under the guidance of four central bodies (NDRC, MIIT, the Cyberspace Administration, and MOST). Kind of like the Frontier Model Forum, but broad rather than frontier-focused: there were 1,300 members (various AI orgs) in mid-2025. It set up a dedicated Safety and Security Governance Committee in late 2023, which develops specifications that CAICT’s evaluations test against. It also issued a voluntary AI Safety Commitment (人工智能安全承诺) now signed by 22 foundation model developers.
International representation
CnAISDA (China AI Safety and Development Association, 中国人工智能发展与安全研究网络): the “Chinese AISI,“ launched in February 2025. CnAISDA is a loose network of existing Chinese institutions: Tsinghua (the main institution, according to Carnegie), CAICT, Shanghai AI Lab, BAAI, and CCID, an MIIT-linked industrial body.
CnAISDA’s primary function is outward-looking international representation - providing China with a seat at the table of AISIs.
According to Carnegie, its network structure lets Beijing avoid designating a single dominant institution for safety and development, which preserves flexibility; and government backing likely reflects aspirations for global participation more than concern about catastrophic risk. It has yet to convert international engagement into domestic safety policy.
WAICO (World AI Cooperation Organization): a Shanghai-based intergovernmental organization for civilian AI capacity-building, governance and standards, consisting of 29 states (BRICS, Global South countries). Proposed by Premier Li Qiang at the opening of the 2025 World AI Conference and formally constituted at this year’s WAIC.
Chinese academia
Much of China’s AI safety work currently happens in academia rather than inside labs. For a systematic overview of the output, see Concordia AI’s excellent State of AI Safety in China (2026) report.
Note that most Chinese safety researchers are not “AGI-pilled” in the American lab sense: they do not generally expect LLM scaling to produce very powerful systems very quickly, and they do not treat LLMs as a categorically exceptional technology. Autonomous vehicle safety, for instance, is often discussed in the same breath as LLM safety.
But, as Zilan has previously discussed, our Chinese counterparts do not need to be AGI-pilled to be safety-pilled. Though the conceptual framework around AI differs from the American one, Chinese researchers and institutions have become acutely aware of a wide range of risks — and Chinese models certainly pose real safety challenges of their own!
A few technical research groups of note include:
Shanghai AI Lab: a state-backed “New R&D Institute” (NRDI) spearheading AI safety research. In the past year, it co-authored ~1 in 10 safety papers coming from China.
Tsinghua University
Prof. Andrew Yao, Dean of the College of AI and Institute for Interdisciplinary Information Sciences (IIIS), cares about safety. He has spoken at IDAIS, and his group at IIIS works on many AI safety research projects.
Tsinghua also recently launched a “Beijing Constellation”: the Qingzhi AI Safety Hub. Qingzhi is hosting the Beijing AI Safety Residency, a 3-month, in-person program beginning this fall!
They may run future iterations of the residency, and also have other ambitious plans – sending more Chinese students abroad to programs like MATS and Astra, hosting speakers and other fieldbuilding projects, etc.
I attended the opening ceremony at Tsinghua and was extremely excited! I encourage you to visit and apply for future cohorts.
Fudan University: FudanNLP, FVL Lab, and the System Software and Security Lab have published work in deception, oversight, backdoor attacks, and red-teaming.
Peking University: A notable group is Prof. Yang Yaodong’s Center for AI Safety and Governance, where I was a research collaborator back in my technical alignment research days.
Beijing Academy of Artificial Intelligence (BAAI, 北京智源人工智能研究院): among the first Chinese institutions to engage seriously with frontier risk. At the 2023 BAAI Conference, it co-organized an AGI safety forum with Concordia AI – the first of its kind in China!
Beijing Key Laboratory of Safe AI and Superalignment
Shanghai Jiaotong University
In AI governance, there is a community of IR and legal scholars that studies frontier model governance and participate in various Track 1.5 / Track 2 dialogues. For some color into Chinese IR x AI perspectives, I recommend:
ChinAffairs+, written primarily by Chenghao Sun, a fellow at Tsinghua’s Center for International Security and Strategy (CISS), and with a variety of other contributors.
SinoAI Insights, a podcast run by Dean Xiao Qian, Deputy Director of CISS.
AI companies
A nontrivial number of Chinese AI company employees care about frontier safety. During my China visit, I had conversations with ~10 current or previous employees of various labs about catastrophic and existential risks from loss-of-control and misuse.
In addition, most major Chinese labs have official safety/security teams (for instance, Alibaba’s, which recently released a thorough model specification for Qwen) and affiliated research institutes (e.g. Ant Group Research Institute, Tencent Research Institute), which independently publish safety and security research papers.
However, Chinese AI companies’ safety practices continue to lag behind those of Western counterparts; Chinese AI companies can be resource-constrained (in compute, money, and capacity), and internal lab structures can make safety advancements difficult.
AI safety organizations
Concordia AI, a Chinese non-profit which advises labs and government bodies on AI safety in China – and recently released the 2026 version of their excellent State of AI Safety in China report.
Safe AI Forum, a US non-profit which runs the International Dialogues on AI Safety, works on research and workshops in AI governance, and advises other China-facing projects.
Open Community for AI Safety China (AI安全开放社区), which runs various online social channels and a Chinese BlueDot course, and is beginning to do more in-person events. Follow them on Zhihu or Bilibili!
SaferAI works on risk management and evaluations for Chinese AI companies. I’ve worked here for the past few months.
Neo Research and Singapore AI Safety Hub (SASH): two other East-facing organizations, not based in China but deserving of a shout. Neo Research focuses on evaluations and novel safety research for Chinese models, and SASH has an awesome team working on verification prototypes. Both are hiring talented engineers.
Various philanthropic organizations are excited about funding projects that support international coordination and China’s AI safety scene.
Startups
Two AI safety startups of note that are operating in China:
Fangcun Leap, spun out of IIIS at Tsinghua. Fangcun sells products for AI agent observability, automated red-teaming, and safety evaluations, similar to Gray Swan AI. Its founders are quite safety-pilled and, based on my conversations with them, interested in other interventions such as pretraining data filtering and input-output classifiers.
Whitzard (白泽), a commercial AI security venture that funds a technical safety research wing, Nuwa Frontier AI Safety Lab. Whitzard and Nuwa were both founded by Prof. Xudong Pan at Fudan University and explicitly complement each other – research from Nuwa supports work like AgentGuard.
On engaging with China’s AI safety scene
Clearly, AI safety and governance in China will be crucial to making AI development go well. If you are a researcher, funder, or AI safety enthusiast — here are several promising areas of work you should consider:
Do technical AI safety research on Chinese models
Chinese models and AI applications probably raise a different set of safety risks from their Western counterparts. Open-weights release is one clear difference. But in addition, Chinese labs may have different exposure to loss-of-control risk due to different internal deployment setups; downstream applications in the domestic market carry their own risk profile; and the regulatory environment creates a distinct set of incentives and constraints.
Safety research from Western labs may therefore fail to capture important risks. Consider working on:
empirical safety research focused on Chinese models (interp, alignment, scalable oversight, etc; for instance, how eval-aware are they?);
more public, independent evaluations of Chinese models.
Neo Research is thinking a lot about this!
Research open-weight model safety
Chinese models are consistently released open-weight, but we don’t have robust OS safeguards and there is relatively little OS model safety research generally.
If you’re technical, consider researching open-weight model safety, possibly in collaboration with Chinese researchers, and producing findings that Chinese labs can ultimately fold directly into their training pipelines.
A heuristic from my time as a technical AI safety researcher: because AI companies everywhere have relatively low willingness-to-pay for safety, try to make adoption of your research as frictionless as possible. E.g., publish off-the-shelf artifacts that can be easily dropped into existing training pipelines, with open-sourced code, clear documentation, and testing on Chinese models.
(And of course, on the oft-asked question of why Chinese AI companies even open-weight in the first place: Jason and Sophie propose a few reasons in this post, which I endorse!).
Support the third-party auditing ecosystem
Independent frontier AI auditing is the near-term governance mechanism I am most excited about! Audits and evaluations could be important for ‘the scramble,’ and also support more ambitious (“fancy”) verification regimes down the line. I’ll write more about this soon.
Consider working on auditing for Chinese AI development. SaferAI is doing excellent work in third-party evaluations, and is currently hiring Research Scientists and Engineers and a Founding Infrastructure Engineer.
Create better Chinese-language safety resources and programs
There is still relatively little high-quality Chinese-language coverage of AI safety research and ideas, which prevents a broader Chinese audience from seriously engaging with the ideas.
If you have native-level Chinese and strong AI safety context, consider the following:
Write accessible, original explanations of important safety research and debates.
Translate AI safety texts, e.g. the Intelligence Curse, into Chinese.
Build Chinese-language versions of programs such as BlueDot courses, SPAR, or MATS.
Run reading groups, research fellowships, or technical upskilling programs for Chinese audiences.
Existing work includes the Open Community for AI Safety China and its Chinese-language AI safety course, as well as Michael and Yilin’s Fall 2026 SPAR project on AI safety in Chinese.
Engage with Chinese academic research
If you’re a researcher, consider collaborating on research projects with Chinese academics – after all, durable international cooperation is built on human relationships :)
Also consider PhDing or joining faculty at Chinese universities with AI safety labs, e.g. Tsinghua’s College of AI!
Work on AI verification for international governance
If you want to engineer technical verification systems, consider joining the Singapore AI Safety Hub, which is building prototypes to internationalize AI verification. SASH is currently hiring a Technical Lead and Research Engineers - open roles here!
If you are interested in fieldbuilding for technical verification in Asia, keep an eye out for future iterations of programs like the Frontier AI Security Residency and Hardware Assurance Programme. Sabrina Shih and I are mentors for the current FASR cohort!
For more non-China-specific project ideas, see The Compute Verification Post.
Notes
There are various skills that might make you better suited to engaging China’s safety ecosystem: e.g., strong understanding of frontier safety risks, Chinese language skills, cultural context.
I’d advise caution when deciding whether and how to engage with AI safety counterparts in China. Western assumptions about AI safety and institution-building do not always transfer cleanly to the Chinese context, and well-intentioned projects can end up being counterproductive. China-facing cooperation may also narrow your options for pursuing other AI safety paths in the future.
Before getting started, I recommend speaking with trusted people who have experience navigating Sino-Western cooperation (e.g. SAIF).
But with those caveats, I hope you think carefully about what’s most useful for you to do in the world - it may very well be engaging with Chinese AI safety!
Thanks to many Chinese researchers and AI safety-ers for conversations during my trip to Asia, and to — among others — Celeste Li, Andy Wang, and Luke Drago for helpful feedback on drafts of this post. I am writing in a personal capacity, and all errors are, of course, my own.
thanks for writing this!