3. Pointing It at What Is on Your Screen
Screenshots send pixels. Work with Apps sends the actual text out of your terminal or editor. The difference matters most exactly when you need it most.
The WJS Desk
Sep 16, 2026 · updated 8 days ago · 4 min read
Everything so far has involved you typing or pasting. This lesson is the desktop app's real advantage over a browser tab: it can see what is open on your machine, so you stop copying and pasting entirely.
Three different things, often confused
People use "it can see my screen" to mean three separate mechanisms with different permissions and different reliability.
| Mechanism | What it does | Needs |
|---|---|---|
| Attach a screenshot | Sends a picture of a window or region | Screen recording permission |
| Work with Apps | Reads the actual text out of a supported app | Accessibility, plus Apple Events |
| Drag a file in | Sends the file itself | Nothing |
The middle one is the interesting one, and the difference matters. A screenshot is an image, so the model reads pixels and can misread them. Work with Apps pulls the underlying text through macOS's Accessibility API, so what arrives is characters, not a picture of characters. On a terminal full of stack traces that distinction is the difference between useful and nearly useful.
Turning on Work with Apps
This is where you grant the Apple Events permission we declined in lesson 2, now that you can see what it buys.
- Open the app's settings and enable Work with Apps.
- Grant Accessibility when macOS asks. System Settings, Privacy and Security, Accessibility.
- Grant Automation for the specific apps you want it to read. macOS asks per application, which is the good design here: you are not granting "all apps", you are granting Terminal, or VS Code, one at a time.
- Open one of the supported apps and check for the banner over the input bar naming the app and what it has picked up.
Supported, as documented by OpenAI: VS Code, Xcode, Terminal, iTerm2, TextEdit, Notes and Script Editor. VS Code additionally needs an extension installed. If your editor is not on that list it will not work, and no amount of permission granting changes that.
The banner is the feature. Before you send anything, the app shows you which application it is reading and what content it has. Read that banner. It is the only moment you get to notice that it has attached the wrong window, and once you send, the content has gone.
What this is actually good for
Four jobs where it beats copy and paste, in rough order of how often we reach for it.
An error in a terminal. The classic. Something failed, the output is forty lines, and selecting exactly the right forty lines with a mouse is annoying enough that you often do not bother.
What went wrong here and what is the first thing
to try? Do not suggest reinstalling anything until
you have explained what the error means.A document you are in the middle of. "Read what I have written so far and tell me which paragraph is doing the least work." No pasting, no losing formatting.
Something you did not write. A config file, a script a colleague left, a document you inherited. Everything in our lesson on asking questions about your own documents applies, minus the paste step.
Comparing what is on screen to what you meant. "Here is the spec I was given, in the other window. Does what I have written actually do that?"
Screenshots, and when they are the right tool
Screenshots are worse for text and better for everything else. Use them when the thing you care about is visual rather than textual:
- Layout problems. "This is misaligned, why." No text extraction helps.
- Charts and dashboards. The numbers are drawn, not written.
- Apps not on the supported list. Your accounting software, a design tool, anything in a browser.
- Error dialogs. Often not selectable text at all.
The launcher can capture a specific window rather than the whole screen, which matters more than it sounds: a full desktop screenshot sends every other window you had open, including whatever is in them.
What we would not point it at
This is the lesson's real warning, and it follows directly from how the feature works.
Work with Apps sends the content of the window when you send a message. Not a summary, not the selection: the content it shows in the banner. So think about what is in that window first.
- A terminal with credentials in the scrollback. A key you exported an hour ago is still up there.
- A config file with secrets in it. Very common, and the point of the file.
- A document with other people's personal data. Their information, your decision.
- Anything under an NDA that says where it may be processed.
The habit that makes this safe. Clear the terminal before you attach it. One command, clear, then rerun the thing that failed. You send the error and nothing else. This costs two seconds and removes the entire category of problem, and it is the only advice in this lesson we would call mandatory.
Turning it off
Per app, which is the useful granularity: System Settings, Privacy and Security, Automation, and untick the specific application. Or remove ChatGPT from the Accessibility list to disable the whole feature at once. Everything else in the app keeps working.
Before the next lesson
Turn it on for one app only, whichever one you hit errors in most, and use it on the next real error you get. One app is enough to judge whether this changes how you work.
Next: talking to it instead of typing, and the difference between two voice features that look the same and cost very different amounts.
