Skip to main content
SAICA-KG

Papers

AgentBay: A Hybrid Interaction Sandbox for Seamless Human-AI Intervention in Agentic Systems

Y. Piao, Hongbo Min, Hang Su, Leilei Zhang, Lei Wang, Yue Yin, Xiao Wu, Zhe Xu, Liwei Qu, Hang Li, Xinxin Zeng, Wei Tian, Fei Yu, Xiaowei Li, Jiayi Jiang, Tongxuan Liu, Haoxiang Tian, Yufei Que, Xiao-Qing Tu, Bing Suo, Yuebing Li, Xiang-Hui Chen, Ze-Yu Zhao, Jia-Xiang Tang, Wei Huang, Xuguang Li, Jing Zhao, Jin Li, Jie Shen, Jinkui Ren, Xiantao Zhang · 2025 · arXiv

Source →arXiv:2512.04367DOI: 10.48550/arXiv.2512.043671 citations
agentic-aisupervisionMCPbenchmarkguardrails

TL;DR

It is proposed that AgentBay provides a foundational primitive for building the next generation of reliable, human-supervised autonomous systems, especially under poor network conditions.

Abstract

The rapid advancement of Large Language Models (LLMs) is catalyzing a shift towards autonomous AI Agents capable of executing complex, multi-step tasks. However, these agents remain brittle when faced with real-world exceptions, making Human-in-the-Loop (HITL) supervision essential for mission-critical applications. In this paper, we present AgentBay, a novel sandbox service designed from the ground up for hybrid interaction. AgentBay provides secure, isolated execution environments spanning Windows, Linux, Android, Web Browsers, and Code interpreters. Its core contribution is a unified session accessible via a hybrid control interface: An AI agent can interact programmatically via mainstream interfaces (MCP, Open Source SDK), while a human operator can, at any moment, seamlessly take over full manual control. This seamless intervention is enabled by Adaptive Streaming Protocol (ASP). Unlike traditional VNC/RDP, ASP is specifically engineered for this hybrid use case, delivering an ultra-low-latency, smoother user experience that remains resilient even in weak network environments. It achieves this by dynamically blending command-based and video-based streaming, adapting its encoding strategy based on network conditions and the current controller (AI or human). Our evaluation demonstrates strong results in security, performance, and task completion rates. In a benchmark of complex tasks, the AgentBay (Agent + Human) model achieved more than 48% success rate improvement. Furthermore, our ASP protocol reduces bandwidth consumption by up to 50% compared to standard RDP, and in end-to-end latency with around 5% reduction, especially under poor network conditions. We posit that AgentBay provides a foundational primitive for building the next generation of reliable, human-supervised autonomous systems.

Editorial notes

Introduces AgentBay, a sandbox service providing isolated execution environments across Windows, Linux, Android, web browsers, and code interpreters with a hybrid AI-or-human control interface (MCP / SDK programmatic access for agents; seamless human takeover via an Adaptive Streaming Protocol). Reports >48% success-rate improvement on complex tasks for the Agent+Human model. Direct evidence that HITL-with-low-friction-takeover is a measurable supervision lever. Relates to SAICA-KG failure modes: incomplete_execution, scope_creep. Useful as a Tool-node citation for the sandbox category and as evidence that the human-takeover latency budget is itself a supervision design parameter (echoing Lindner-2025's sync/semi-sync/async axis).

View raw YAML on GitHub →