AI安全威胁模型与对抗性评测

AI Safety and the Race to Demonstrate Threat: How AI Labs Compete Over Danger — a survey of a frontier of autonomous AI in the Agents in the Wild series.

Book framework (chapter themes):

• Introduction — The AI Safety Threat Landscape and the Scope of This Survey
• Core Concepts and Theoretical Frameworks of AI Safety
• Formal Formulation of the Alignment Problem and Its Theoretical Difficulties
• Taxonomy and Threat Modeling of Large Language Model Safety Threats
• Jailbreak Attacks and Prompt Injection: Mechanisms, Variants, and Detection
• Safety Threats of Multimodal Models and Cross-Modal Attack Surfaces
• Agent Safety: Tool Use, Autonomous Action, and Runtime Protection
• Post-Training Optimization of Safety Alignment and Fine-Tuning Risks
• Inference-Time Safety Scaling and Computational Efficiency Trade-offs
• Construction, Classification, and Consistency Analysis of Safety Evaluation Benchmarks
• Threat Assessment for Physical Safety and Embodied Intelligent Systems
• Adversarial Machine Learning: Attack Mechanisms and Defense Strategies
• Application of Mechanistic Interpretability in Safety Verification
• Methodology of Red Teaming and Threat Demonstration Competitions
• Safety Guardrails and Runtime Monitoring Mechanisms
• Safety Threat Modeling for Human Oversight and Socio-Technical Systems
• Security Threats of Cyber-Physical Systems and Autonomous Driving
• Existential Risk and Safety Governance of Superintelligence
• Frontier Challenges and Open Problems in Safety Evaluation
• Future Directions: From Threat Demonstration to the Safety Engineering Paradigm

Who it is for. Researchers, engineers, and advanced enthusiasts in the field of artificial intelligence who have a foundation in machine learning and security engineering and who wish to systematically understand AI safety threats, defense mechanisms, and evaluation methodologies.