Computer-use AI for legacy systems: when agents beat APIs or RPA

Computer-Use AI for Legacy Systems: When Agents Beat APIs or RPA and When They Don't

Legacy automation often starts with an awkward fact. The software may lack a usable API. Direct database access may be unsafe and replacement could take years.

Computer-use AI offers another route. An agent can observe a browser or desktop interface then choose where to click and what to enter. OpenAI supports this model through hosted browser sessions and controlled desktop environments. Its guidance also calls for isolated execution, action limits and independent outcome checks. (developers.openai.com)

That does not make an AI agent the default choice. The categories also overlap:

• API integration sends structured requests through a documented interface.
• Conventional RPA follows predefined logic using selectors, accessibility trees, OCR or computer vision.
• Computer-use AI chooses actions dynamically from screen content and task context.
• Hybrid automation uses AI for interpretation but keeps validation and execution deterministic.

Modern RPA is not limited to coordinates. For example, Power Automate supports UI Automation, MSAA and raw accessibility selectors for complex or legacy applications. (learn.microsoft.com)

Use measurable selection criteria

Treat this as a scorecard rather than a rule. A high-volume process may still need UI automation when no interface exists. A variable process may be safer when an AI model extracts data and deterministic code completes the transaction.

Where computer-use AI can win

Agents fit tasks where people rely on context rather than fixed navigation. Consider an old ERP receiving order exceptions in several document formats.

A controlled workflow could let the model classify the exception and locate a candidate order. Deterministic checks would confirm the customer ID, order number and permitted correction. The agent would prepare the change but require review below a defined confidence threshold. A separate query would then verify the saved result.

For an illustrative pilot, assume 1,000 cases per month and six minutes of manual work per case. Set acceptance criteria before deployment, such as 98% correct order selection, no duplicate updates and human review below 25%. Stop the pilot if reconciliation finds an unapproved change. These figures are planning assumptions, not general benchmarks.

Feasibility still depends on the interface. Test terminal emulators, Citrix sessions, canvas-rendered screens, custom controls and low-resolution displays. Also test MFA, CAPTCHAs, session timeouts and accessibility-tree availability. If the agent cannot identify state reliably, it should not execute transactions.

The control plane should constrain the agent

Run each agent in an isolated browser or virtual machine. Use least-privilege identities, network allowlists and restricted file or clipboard access. Scan downloads and prevent secrets from entering screenshots or logs.

Screen content must be treated as untrusted. Approval policies should cover payments, deletion, permission changes and sensitive-data submission. The approver should see the source record, intended action, affected account, before-and-after values and policy reason. An agent-written summary alone is not enough.

Each transaction also needs:

• Precondition and postcondition checks
• Idempotency keys and duplicate prevention
• Limited retries with clear failure categories
• Session and concurrency controls
• Compensating actions for reversible errors
• Reconciliation against the system of record
• Time, step and cost limits
• Complete audit evidence

NIST recommends documented oversight, change management and risk-based human review for generative AI systems. (nvlpubs.nist.gov)

Calculate cost before choosing the architecture

Do not assume agents are cheaper at low volume or APIs are always cheaper at scale. Estimate:

TCO = build cost + licenses + model use + virtual desktops + monitoring + review labor + failure recovery + maintenance

Then divide by verified successful transactions over the expected solution life. Compare that figure with manual handling, RPA, API development and incremental modernization.

Lifecycle costs matter too. Keep representative test cases, golden task recordings and sandbox environments. Version prompts and policies. Pin model versions where possible then use canary releases and regression tests after model or UI changes.

Use computer-use AI as a controlled bridge

Prefer APIs when reliable interfaces provide the required coverage. Prefer deterministic RPA when rules and screens remain stable. Consider computer-use AI when tasks are visually accessible, variable, reversible and compatible with human review.

Modernize when automation maintenance compounds strategic technical debt. An API layer or staged modernization roadmap may reduce long-term risk without requiring a full replacement.

Webalize's Automation & AI Implementations service helps teams apply this vendor-neutral checklist, test feasibility and build controlled automation around existing software. The goal is not to force a visual agent into every gap. It is to choose the least complex architecture that can meet the required controls.