Skip to content
Superintelligence Conferences

Topic

AI Safety and Governance

Alignment, risk, oversight and the rules being written around advanced AI.

Alignment: the core technical problem

Alignment is the challenge of ensuring AI systems pursue the goals their designers intend. Stuart Russell argues that systems optimising a fixed objective become more dangerous as they become more capable, and that machines should instead remain uncertain about human preferences.

Safety research also covers robustness, evaluation of dangerous capabilities, interpretability and monitoring.

The emerging rulebook

The EU AI Act entered into force on 1 August 2024. Prohibited practices and AI-literacy obligations applied from 2 February 2025, obligations for general-purpose AI models from 2 August 2025, and most remaining rules from 2 August 2026. The AI Omnibus, in force since 27 July 2026, set two later dates for high-risk systems: Annex III high-risk rules apply from 2 December 2027, and high-risk rules for AI embedded in regulated products (Annex I) from 2 August 2028 (European Commission, digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai, verified 5 October 2026).

Internationally, the International AI Safety Report (2025), chaired by Yoshua Bengio, synthesised the scientific evidence on the capabilities and risks of general-purpose AI for governments.

Business implications

  • Deployers, not only developers, carry obligations under the AI Act.
  • AI literacy for staff is now a legal expectation in the EU.
  • Incident reporting and documentation become routine governance tasks.

Questions to ask

  • Have we inventoried the AI systems we develop and deploy?
  • Which staff need AI-literacy training?
  • How do we evaluate a vendor's safety practices?

Insights

Articles on this topic are in preparation.

Speakers and thinkers

Sources

  1. S. Russell, Human Compatible (2019) — author page — verified 2026-10-05
  2. European Commission — AI Act regulatory framework and application timeline — verified 2026-10-05
  3. Regulation (EU) 2024/1689 (AI Act), Official Journal — verified 2026-10-05
  4. International AI Safety Report (2025), UK Government publication — verified 2026-10-05
  5. NIST AI Risk Management Framework — verified 2026-10-05