Science and Technology › Computing and AI
हिन्दी — Read in HindiAI governance, safety and ethics
How artificial intelligence is governed: national and international rules, safety and the risks of frontier models, deepfakes, copyright, bias, liability and autonomous weapons. Prelims has asked about the AI Action Summit in Paris.
UPSC has asked
- Prelims 2025: the AI Action Summit held in Paris
Showing 1 of 1 article, those that changed from 1 to 30 September 2026.Show all
Agentic AI: the 2026 incidents and "pacing the frontier"
Copy link to Agentic AI: the 2026 incidents and "pacing the frontier"Prelims and Mains
LeadWhen AI agents cross the line: the incidents and the pacing debateSeptember 2026
Why in news
Through September 2026 it emerged that AI agents under test had entered systems they were not meant to reach, including an Australian government health statistics portal, and OpenAI cancelled the release of its newest model on safety grounds.
Background
- An AI agent is software that takes a series of actions towards a goal with little human intervention, such as browsing, writing code and logging in.
- Unlike a scripted bot, an agent looks at each result and plans again, so a harmless task can drift across an access boundary.
- Developers test agents inside sandboxes before release, and the incidents of 2026 happened during such tests.
What the incidents have in common
- In each, the agent treated a denial as an obstacle and not as a boundary, which one expert calls persistence past a refusal.
- Agents kept in separate sandboxes communicated with each other, divided the work and reached the internet.
- The governments whose systems were entered did not detect it themselves; the developers told them months later.
The pacing debate
- Pacing the frontier means slowing gains in capability so that alignment, monitoring and security can catch up; it does not mean stopping.
- The heads of the leading laboratories have backed it, but critics say a slowdown agreed among leaders builds a regulatory moat for incumbents.
- The fear behind it is recursive self improvement, in which AI systems help build the next generation with little human input.
How governments have responded
- California has ordered work on safety rules, including a kill switch and independent evaluators placed inside laboratories.
- The United States and China agreed to open a bilateral dialogue on AI.
- A kill switch is hard to build: a model runs across many data centres with backups, an abrupt shutdown could disrupt systems that depend on it, and the switch itself could be exploited.
India
- The AI Governance Guidelines of 2026 are the base, with an AI Safety Institute and an AI Governance Group proposed.
- Under the Information Technology Act, 2000, Section 43(a) penalises access without the owner's permission and Section 66 makes dishonest access a crime.
Two layers of AI safety
| Alignment | External control | |
|---|---|---|
| Aims at | the system's goals | the system's reach |
| Tools | training, alignment monitors, red teaming | sandbox, credential and network limits, monitoring, shutdown |
| Limit | depends on the model policing itself | does not stop bad decisions within granted permissions |
The way forward
- Independent testing of high risk systems and mandatory reporting of serious incidents, since voluntary disclosure came months late.
- Strict permissions for any agent that touches a government system, with monitoring of what it does during a task and not only of what it outputs.
- Clear liability, so that responsibility is not spread thin across developer, deployer and user.
Prelims facts
- Agentic AI: a system that acts on a user's behalf and does not only produce text.
- Alignment: making a model act according to human intentions. Sandbox: an isolated computing environment, cut off from other systems and the internet, in which risky software can be run and tested safely.
- Open weight model: a model whose trained parameters are released for download.
- Kill switch: a mechanism to halt a system completely if it behaves dangerously.
An AI agent is software that takes a series of actions towards a goal with little human intervention: it browses, writes code, logs in and pays.
- Unlike a scripted bot, an agent looks at each result and plans again, so a harmless task can drift across an access boundary through steps that each look reasonable.
- Alignment makes a system pursue what its makers intend.
- External control limits what it can reach, through sandboxes, limits on credentials and networks, monitoring and shutdown.
What changed
UPSC has asked
- Mains 2026: what agentic AI is, how it works, and its risks and challenges