Desktop Automation Assistant

import { CardGrid, LinkCard, Steps, Badge } from ‘@astrojs/starlight/components’;

After reading, you can: Issue natural language commands to let AI operate mice, keyboards, browsers, and Windows applications automatically.

1. What It Can Do

Scope	Underlying Tool	Practical Example
🌐 Web & Browser	`browser-harness` Chrome controller	”Open Google News, extract the top 5 articles, and format them into a markdown table.”
💻 Desktop & Apps	9 Local Native Commands (`desktop_automation_*`)	“Open Word, create a document titled ‘Q3 Summary’, and type the meeting minutes." "Empty the Recycle Bin and minimize all windows.”

2. 3-Step Setup

Select the Assistant
Start a new chat and choose Desktop Automation from the assistant dropdown.
Enable Remote Debugging in Chrome (First time only)
Go to chrome://inspect/#remote-debugging in Chrome and enable remote debugging.
Verify Status
In Settings → HuiAI → Automation Status, click Test Connection to confirm both services are .

3. Safety and Emergency Stop

🛡️ User Approval: Actions involving payment, deletion, submission, or sensitive data trigger an approval modal.
🛑 Emergency Stop: Clicking the Stop button halts both AI text generation and low-level mouse/keyboard operations immediately.
📺 Visual Oversight: Combine with the Screen Wall panel to observe actions live.