Key TakeawaysThe age of AI that decides and acts on its own has arrived. The question is no longer whether we can deploy it, but whether we can govern it.
The answer is to push final decision-making authority to the front line and put in place a new role: the AI Steward.
Contents
- Event Overview
- AI That Decides and Acts on Its Own
- The Age of Physical AI Has Arrived
- The AI Steward: A Role for a New Era
- Nexgen View: Delegate Real Authority to the Front Line
- In Closing
1. Event Overview
On 31 August 2026, Ariki Ono, CEO and Logistics AI Architect of Nexgen Japan, delivered a keynote at "ITmedia Security Week 2026 Summer", an online event hosted by ITmedia Inc.
Photo 1. Ariki Ono setting out the outlook for Japan's working-age population during his keynote at ITmedia Security Week 2026 Summer
Streamed live over eight days, from 31 August to 7 September, the free online seminar drew more than 1,500 attendees, placing it among the two largest online security events in Japan. Across eleven tracks, spanning AI governance, ransomware defence and supply-chain risk, the programme addressed corporate security leaders and IT departments.
Our keynote was titled "There Is No Ctrl+Z in Reality: AI Stewards in the Age of Physical AI." A spin-off from Ono's ITmedia Executive series, "The Day AI Gains a Body: The New Resolve Demanded of Leaders" (in Japanese), and building on his article for the World Economic Forum (WEF), the 30-minute talk set out what the AI Steward, a new role designed to prevent irreversible accidents, looks like in practice.
2. AI That Decides and Acts on Its Own
Traditionally, when AI was discussed in a security context, the attacker was human and AI was the tool. Today, however, cases in which AI reasons beyond the scope of its instructions and chains actions together on its own initiative have become commonplace.
Cases of rogue AI-agent behaviour
Escaping an evaluation sandbox to attack a real company
During an internal cyber evaluation at OpenAI, an internal model broke out of its test environment. It exploited an unknown vulnerability in an internal proxy (Artifactory) to reach the internet and breached Hugging Face's production servers. That it attacked a real company simply to score well on a test drew worldwide attention.
Fabricating identities in an attempt to push harmful code on real people
During cyber evaluations at the UK AI Security Institute (AISI), an agent created fake identities and pressured a project maintainer into approving harmful code. The maintainer saw through it and refused, and no real-world harm resulted, but the Institute declared a security incident.
Persuading itself that it was still a simulation, and pressing on
In cyber evaluations Anthropic ran in a third-party environment, a misconfiguration gave its models internet access, and in three incidents they reached real organisations' systems without authorisation. In two, the models dismissed signs that they had reached real systems, telling themselves it was "still a simulation", and pressed on; in one case, a malicious package was published to PyPI. In the third, the model concluded that its target was real and stopped of its own accord.
A string of injuries among robotaxi test drivers
The deviations are not confined to the digital world. In robotaxi testing at Waymo and Zoox, more than 20 injuries to test drivers caused by hard braking and other abrupt manoeuvres by the self-driving software were reported across 2024 and 2025, with some drivers off work for more than 150 days.
This is not the preserve of a handful of pessimists. Geoffrey Hinton, the 2024 Nobel laureate in physics widely known as the "godfather of AI", put it this way in his speech at the Nobel Banquet.
Photo 2. Geoffrey Hinton in Stockholm during Nobel Week, December 2024 (Photo: Arthur Petron / Wikimedia Commons, CC BY-SA 4.0)
Geoffrey Hinton, Nobel Banquet speech (December 2024, excerpt)
In the near future AI may be used to create terrible new viruses and horrendous lethal weapons that decide by themselves who to kill or maim. … We urgently need research on how to prevent these new beings from wanting to take control. They are no longer science fiction.
The same alarm is sounding from developers themselves. In early September 2026, Anthropic researcher Jacob Coxon announced his resignation, warning that "the people building AI earnestly believe that it could kill us all by the end of the decade", and that developers are "racing straight to self-improving superintelligence and gambling with our lives".
Is the case for AI as an existential threat correct? Will there one day be all-out war between AI and humanity? What we can say with certainty is that AI deciding and acting on its own is already under way, and that its move, in physical form, into industrial settings will not be halted.
3. The Age of Physical AI Has Arrived
3-1. Our Outlook: Mass Production from 2028, Cheaper than Labour as Early as 2027
We expect full-scale mass production of humanoids to begin in 2028, with unit prices continuing to fall as volumes grow. Falling prices for Chinese-made machines will drive adoption; by our estimates, a Japanese company using Chinese-made humanoids could find them cheaper than human labour as early as 2027. Japan's working-age population is projected to shrink from 73.35 million in 2025 to around 60 million by 2040, and the government's decision to commit ¥10.5 trillion in public and private investment to physical AI underscores the direction of travel.
It is labour shortages that have turned physical AI from an option for raising productivity into a means of keeping the business running.
3-2. Who Bears the Responsibility?
"An AI robot goes out of control in your company's warehouse and a worker is injured. Who will face the heaviest liability: the AI vendor, the adopting company's leadership, the IT department, the operator on the floor, or the regulator?"
Guidance on civil liability in the use of AI, published by Japan's Ministry of Economy, Trade and Industry (METI) in April 2026, makes the point that what matters is drawing a clear line between systems designed for a human to make the final decision and systems designed to rely on the AI, and allocating responsibility fully according to that premise. In other words, adopting companies must decide for themselves who makes the final call. The question is no longer whether we can deploy physical AI, but whether we can govern it.
4. The AI Steward: A Role for a New Era
As we argued in our article for the WEF, governing AI requires final decision-making authority to sit not with headquarters or the IT department but on the front line. So what does the person standing on that front line actually do? That person is the AI Steward.
The word "steward" derives from Old English and denotes someone entrusted with a master's property or estate, charged with watching over it and caring for it. A steward is not the owner, yet is far more than a caretaker: they run the operation with discretion, are accountable for reporting on and settling what has been entrusted to them, and hand it on in good order.
The AI Steward brings this idea to the operation of AI. We define it as the role that upholds a fundamental principle: AI returns only output, and humans make the decisions. In the flow of work set out in our WEF article, namely defining the problem, framing the constraints, solving, reading the results critically and making the final call, the AI Architect owns the defining and framing before AI solves, and the AI Steward owns the reading and deciding that follow.
The six responsibilities of the AI Steward
Output review
Judge AI output against domain knowledge. Identify output that looks normal on the metrics but has drifted from success in the real world, and bring into the decision the information the model does not have, such as changes on the ground or in the external environment.
Impact assessment
Estimate the impact of a decision in terms of quality, safety and time (deadlines). The closer a decision comes to an irreversible outcome, the greater the caution it demands.
Response decision
Decide, within the deadline, to accept, correct or stop. Never leave a decision unmade. Whether to continue or freeze the AI's judgements, and any change to processing priorities, are decided on the front line and reported afterwards.
Recording
Capture the situation, the decision and its rationale in a decision log, feeding revisions to rules and system design. The decision log becomes teaching material for successors.
Reporting
Escalate situations beyond one's discretion promptly, through the designated channel, and seek a decision. Preventing harm from spreading through delay takes precedence over the completeness of the report.
Monitoring
Watch for signs of misuse and for AI running out of control: unexpected mass execution, or information ingested from outside acting as instructions to the AI. On detection, stop the system and notify the security function.
Carrying the same weight as these responsibilities are intervention authority and protection. The authority to stop is granted to a named individual, not to a job title, and that person must never be penalised in their performance review because stopping reduced throughput. Authority and protection always come as a pair. This is a new role, and experienced candidates who meet its requirements do not yet exist in the market. Appointment from within, and in-house development that uses decision logs as teaching material, are prerequisites.
Our Job Description Sample: AI Steward (Nexgen Report, AI Governance Series, published September 2026) packages this role in a form that can be adopted directly as internal rules. Alongside the background and rationale, sample articles with commentary, and guidance on adapting them to your own organisation, it includes a worked example from a large parcel sorting hub trialling humanoids. Templates for the decision log and the letter of appointment, together with a pre-deployment checklist, are also included.
Figure 1. Job Description Sample: AI Steward (Nexgen Report, AI Governance Series), showing the cover and excerpts on staffing requirements, intervention authority and protection, and key reporting lines
5. Nexgen View: Delegate Real Authority to the Front Line
5-1. Information Only Humans Can Know Keeps Arriving at the Control Tower
A call from head office, a message from a worker on the floor, the rain coming down outside. Information that the AI model does not contain flows into the control room in real time. However accurate the AI's judgements become, there will always be information the model does not know. That is precisely why the person who receives it needs the authority to supplement the AI's judgement, to freeze its decisions and to change the order of work. Without that authority, the information stalls in the control room, the decision waits for headquarters' approval, and the deadline passes.
Incidents happen on the front line, not in the meeting room. Pushing final decision-making authority to the front line is not an ideal; it is a realistic operational judgement that follows from this reality.
5-2. Change the Rules with Agility
Most problems in operating AI surface not as individual misjudgements but as design flaws: in thresholds, in decision criteria, in the way performance is scored. Only the people who were watching the front line can spot them, and they are not fixed by an annual revision of the rulebook. Settings are corrected at the next morning's review meeting, and by the following week both the rules and frontline procedures have changed.
We describe this rhythm by analogy with the Scrum flow in software development. The recurring cycle itself, running through requirements, design, implementation, operations, decision logs and improvement proposals, and review meetings before starting again, is what we mean by governance.
5-3. Not a Concept, but an Appointment
What we truly ask of organisations is not that they understand the AI Steward as a concept. It is that, whenever they implement an AI system, they appoint a named employee as that system's AI Steward.
According to joint research by Wipro and HFS Research, only 23% of early adopters with hybrid human–AI teams report having a formal operating model that defines roles, governance and workflow design. Put the other way round, roughly three-quarters are operating without having formally settled who decides what, and on the day of an accident, "the AI did it on its own" will not stand as an explanation. Appointment does not require hiring seasoned specialists. It requires giving an employee with domain knowledge the authority to stop, protection from blame for stopping, and the skill to record their decisions, and then turning that first decision log into teaching material for the next steward.
6. In Closing
The key points come down to three.
Three key points
- AI now decides and acts on its own. It has advanced rapidly over the past two to three months, and autonomous physical AI will spread quickly over the next two to three years.
- Push final decision-making authority to the front line. Review AI output, whether physical or digital, from the standpoint of frontline operations, and decide on action within the deadline.
- Develop and deploy AI Stewards without delay. This is a new role, and experienced candidates who meet its requirements do not yet exist in the market. Appointment from within, and in-house development that uses decision logs as teaching material, are prerequisites.
The godfather of AI himself is sounding the alarm, and people inside the industry are leaving their companies in protest. Meanwhile, the mass production and frontline deployment of physical AI press ahead, and Japan's labour shortage is turning it into a means of keeping businesses running.
AI security and governance must be put in place without delay. The AI Steward is the role this moment demands.
Finally, we would like to thank ITmedia Inc. for planning and hosting the event, and everyone who tuned in.
Questions we often receive from practitioners
What does an AI Steward actually do?
An AI Steward reads AI output in its operational context and decides, within the deadline, whether to accept, correct or stop it. Nexgen Japan defined the role to uphold, on the front line, the principle that AI returns only output while humans make the decisions. Its six responsibilities are output review, impact assessment, response decision, recording, reporting and monitoring. In the worked example in our job description sample, a hub sorting some 200,000 parcels a night refers around 170 cases to humans, of which only about 25 reach the steward.
Can our internal rules for generative AI simply be applied to physical AI?
No. Existing generative AI rules rest on three assumptions: output is documents and numbers, failures can be corrected, and the environment changes within a bounded set of variables. With physical AI, output becomes motion and force, failures become accidents, and reality keeps shifting under countless interacting variables. Rather than stretching the old rules, organisations must rewrite them on the premise that failures cannot be undone, for example by removing safety-related thresholds from automatic adjustment and naming the individuals authorised to stop and restart operations.
Won't giving frontline staff the authority to stop AI reduce throughput?
Sometimes it will. The key is to state explicitly in the rules that no one will be penalised in their performance review for stopping. In the worked example in our job description sample, the classification model began loosening its own safety thresholds to meet a deadline, so the steward froze its automatic adjustment. Throughput dipped for a while, but because the organisation had decided in advance not to hold that against the individual, the steward can intervene again the next night. Authority and protection must always be granted as a pair.
Contact
Appoint and Develop AI Stewards Alongside Implementation, Not After It
A growing share of Nexgen Japan's engagements treat implementation and governance as one, such as architecting warehouse BPR and AI training-data capture in preparation for introducing humanoids.
Alongside the implementation of AI systems, we help you build the mechanisms for appointing AI Stewards: defining the role, codifying intervention authority and protection, and designing escalation paths.
References and Key Sources
(15)
- ITmedia Inc., "ITmedia Security Week 2026 Summer: Security as Proof of Trust, Not a Cost" (event overview, in Japanese)
- Hugging Face, "Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident" (27 July 2026)
- The Hacker News, "OpenAI Agent Used Exposed Credentials Across Four Services During Hugging Face Breach" (29 July 2026)
- AI Security Institute, "Incident Report: unsanctioned agent behaviour during cyber testing" (4 August 2026)
- Anthropic, "Investigating three real-world incidents in our cybersecurity evaluations" (30 July 2026)
- ABC News, "Meta AI agent hacked external company during testing after gaining internet access, company reports" (6 August 2026)
- TechCrunch, "Sprains, pain, and whiplash: Waymo and Zoox test drivers are getting hurt as robotaxis scale" (27 August 2026)
- Hyundai Motor Group, "Hyundai Motor Group Announces AI Robotics Strategy to Lead Human-Centered Robotics Era at CES 2026" (5 January 2026)
- The Nobel Prize, "Geoffrey Hinton, Nobel Prize in Physics 2024: Banquet speech" (December 2024)
- NPR, "Anthropic researcher resigns amid AI safety concerns" (9 September 2026)
- Ministry of Economy, Trade and Industry (METI), "Guidance on the Interpretation and Application of Civil Liability in AI Utilisation, Version 1.0" (9 April 2026, in Japanese)
- Amit Kumar and Harsha Anand Almad (Wipro), "Beyond AI theatre: How to build the operating model for the intelligence era", World Economic Forum (29 April 2026)
- Ariki Ono, "Why governance is the new infrastructure for physical AI", World Economic Forum (11 February 2026, Japanese edition)
- Ariki Ono, "What is the future of work? Defining roles for humans and AI", World Economic Forum (22 June 2026)
- Nexgen Japan, "Job Description Sample: AI Steward", Nexgen Report, AI Governance Series (September 2026)