Microsoft just dropped a preview feature in Copilot Studio that lets AI agents control your Windows desktop, literally seeing your screen, moving the mouse, and clicking buttons. No API required. No custom integration. Just describe what you want in natural language and watch it happen.
https://learn.microsoft.com/en-us/microsoft-copilot-studio/computer-use
This isn’t traditional RPA (Robotic Process Automation), where you record clicks and element selectors that break every time a UI changes. Computer use is built on Computer Use Agents (CUA), an AI model that combines vision with reasoning. It sees the screen like a human does. When a button moves or a form changes, it adapts—looking for the submit button in its new location rather than clicking coordinates that are now wrong. It doesn’t break like screen scrapers or macros, it figures out the new layout and keeps going.
The use case: Automate the repetitive stuff that has no API. Legacy apps, vendor portals, web forms that change constantly. The tedious data entry and form filling that burns hours. Computer use handles very repetitive tasks.
The security question: Giving AI control of a Windows machine with access to internal systems isn’t something that sits easy with people. Microsoft includes authentication controls, access restrictions, and audit logging, but this obviously needs careful evaluation before any production deployment.
In this post, I’ll set up computer use, test it with a real automation scenario, and explore what enterprise IT needs to know about security controls and limitations.
What can I use them for
Although examples are a bit sparse from Microsoft, here are some from their page, we will try the Bike parts scenario. After trying out computer use agents hopefully you can think of many more scenaios to use them for
Setting Up Computer Use Agents
Prerequisites
Before you start, you’ll need:
- Access to Microsoft Copilot Studio (US region environments only during preview)
- Generative orchestrator enabled on your agent
- Either:
- The hosted browser (quick start, Microsoft-managed), OR
- A Windows machine with Power Automate for desktop v2.59.169.25241 or later
The hosted browser is the fastest way to test, but Microsoft explicitly warns it’s not for production scale, it can be throttled back based on demand. For anything more consuming, you’ll want a dedicated machine.
Step 1)
Region check, fear not if your Microsoft tenant is not in US, as its just the power platform environment that needs to be in the US
Go to https://admin.powerplatform.com > Manage> Environments > New
Configuration:
- Name: Give it something descriptive
- Region: Must be United States for computer use preview
- Make this a managed Environment: For this demo Im saying no but for a secure and goverened environment youd set this to yes
- Type: Choose based on your needs
- Add a Dataverse data store: Yes (required for Copilot Studio)
- Pay-as-you-go with Azure: Optional, depends on your licensing
On the next page, the only thing you’ll need to set is a security group with who can get access. You can even choose None for your demo.
Step 2)
Now in Copilot studio https://copilotstudio.microsoft.com ensure your US environment is selected by choosing it at top right dropdown
Step 3)
We can now create our agent, choose agents > new agent
Step 4)
Again this first one is using Microsofts demo so I’ve chosen (alter as you like)
Click Configure, Next to Describe
Name: Computer Use Agent Demo
Description: Demo for Computer Use Agents
and click Create
Step 5) If you scroll down on your new agent and choose Add tool from tools (You can also select tools on left menu if you were on main page)
Choose, new tool
Choose Computer use
Add Instructions to your agent:
I’m using Microsofts demo for this so
1. Go to https://computerusedemos.blob.core.windows.net/web/Adventure/index.html
2. Submit a new entry for each of the following items:
Rear Derailleur, RD-4821, 50, 42.75, Tailspin Toys
Pedal Set, PD-1738, 80, 19.99, Northwind Traders
Brake Lever, BL-2975, 35, 14.50, Trey Research
Chainring Bolt Set, CB-6640, 100, 5.25, VanArsdel, Ltd.
Bottom Bracket, BB-9320, 60, 24.90, Tailwind Traders
Notice use hosted browser is selected, in this demo this means we will use Windows365 machine hosted by Microsoft. later we can talk about hosting your own machines
Click Add and Configure
We can now add a name and description to the tool (alter as you like)
Name: Submit inventory items
Description: Add products to the inventory system.
Testing Computer Use Agents
Back on the overview page you can test your agent
If you now go to Activity you will see your agent executing
Thoughts and further use cases
You may now be thinking what I am which, is err ok why would I type in all the inventory to have an agent go do it as Ive saved zero time? and this is where I’m sure Microsoft are also aware that the demos on the preview page are a bit more of a, give this a try! but Im sure you can come up with real use cases. I think its targeted at very repeatable processes perhaps things like timesheets or other tasks with no api.
Security and using your own devices
While the hosted browser works for testing, for production use you’ll want your own machines
It’s advisable to use your own machines this could be your laptop or preferably something like an AVD or Windows365 desktop.
Set up your machine with these instructions by Installing Power Automate on them they can then become Agent machines.
Setup Machine for Use : Manage machines – Power Automate | Microsoft Learn
Once you have setup the machines you want to use heres some extra tips on this ensure the machine is in a group and Computer Use for that group is on
Back in CoPilot Studio
In your Agent Got to tools and Machines Click on the 3 dots next to machines to change to your new agent host
Click 3 dots for computer use and add new connection
Connect to a machine and agent end enter your credentials
You will now be running the whole tasks on the machine you have chosen and if thats your local laptop you’ll see that if you re test the add inventory items it’ll do it in front of your eyes.
Securing your endpoints
To secure a lot more I’d advise maybe a vm whether win 365 or AVD, or VM image in Azure
Do:
- Setup a restricted account that is just for running these tasks on the desktop
- Setup Intune security Baseline for this machine so secure it down as much as possible
- Setup Edge Baseline that locks down Edge to only be able to get to websites you’ve defined
- Setup Windows Defender App Control to only allow certain exe’s to be run
- Create restrictive Groups who can use the Automation
Summing Up: The Automation Paradox
Computer Use Agents are in their infancy, but the direction is clear. Just like we’ve seen with other automation technologies, the question isn’t whether they’ll be useful, it’s how we’ll adapt to use them effectively.
Real-World Viability:
The demo scenario highlights the current limitation: you’re not saving time if you’re typing in inventory data to have an agent enter it elsewhere. But that’s the poin, this is preview technology. The real use cases will emerge for genuinely repetitive tasks with no API access: legacy vendor portals, ancient ERP systems, compliance forms that change layouts monthly, timesheet systems that refuse to modernize.
Security Realities:
The security concerns are legitimate. You’re giving AI control of a Windows desktop that could potentially access internal systems. The mitigations I’ve outlined (restricted accounts, Intune baselines, Edge restrictions, Defender App Control) are non-negotiable for production use. Treat these agent machines like you would any high-risk endpoin, because that’s exactly what they are.
Where This Goes:
Website and application testing seems like a natural fit, scenarios where you need consistent, repeatable interactions without the brittleness of traditional Selenium scripts. Data migration from legacy systems. Compliance auditing tasks. The possibilities are there, but each organization will need to evaluate whether their specific pain points justify the investment.
The technology works. The question is whether your use case can justify putting it into production. For many organizations.
References: https://learn.microsoft.com/en-us/microsoft-copilot-studio/computer-use