Alibaba’s Qwen team has introduced a new AI system designed to move beyond traditional chatbots and interact directly with digital devices.
Called Qwen-UI-Agent, the system is a foundation model for graphical user interface agents. Instead of simply responding to a user’s instructions with text, it is designed to carry out actions across mobile devices, computers, websites, and other digital environments. The technical report was published on July 30.
The system combines graphical user interface interaction with command-line execution. This means an agent can potentially use buttons, menus, and applications while also executing commands through a terminal when necessary.
That combination is important because many real-world computer tasks cannot be completed through a single interface. A developer, for example, might need an AI agent to open an application, inspect a file, execute a command, modify a project, and then interact with a web browser.
Qwen’s researchers designed the system around these longer workflows. The agent can perform tasks spanning more than 100 turns, allowing it to maintain context while completing complex operations. The training system also uses more than 10,000 concurrent environments to generate and evaluate agent interactions.
The reported benchmark results are particularly notable on mobile devices. Qwen-UI-Agent achieved 82.1% on MobileWorld, 92.2% on MobileWorld-Real, and 97.5% on AndroidDaily. It also reported a 79.5% score on OSWorld-Verified, a benchmark designed to evaluate agents performing tasks on real computer interfaces.
The project points toward a broader change in how AI assistants may work. Rather than waiting for users to copy information into a chatbot, future agents could potentially operate the same software people already use and complete tasks directly.
Qwen is also experimenting with agents that can proactively initiate services and improve their own capabilities through repeated interaction with digital environments.
The result is a system that looks less like a traditional chatbot and more like an AI worker capable of interacting with computers.
As companies continue developing increasingly autonomous agents, the ability to reliably operate real devices could become just as important as language or reasoning ability. Qwen-UI-Agent suggests that the next generation of AI competition may increasingly focus on what these systems can actually do rather than simply what they can say.
