Show HN: Let Your AI Agents Paint Big Arrows, Boxes And Text On Your Screen

TL;DR

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

A project featured in a Show HN post describes bigarrow, an MIT-licensed macOS command-line tool and agent skill that places temporary visual pointers and labels over app windows. It is designed for moments when an agent needs a person to act or wants to show where to go; it does not click, type or capture screenshots.

A developer has introduced bigarrow, an MIT-licensed macOS command-line tool and skill for Claude Code and Codex that lets AI agents draw arrows, boxes and text labels over open windows. The tool is meant to help an agent direct a person to a button or explain a sequence of steps, while leaving the actual interaction to the user: according to its project page, it does not click, type or capture the screen.

Bigarrow can point to an interface element by its label and app, or to a specified screen coordinate. Its commands can also select a named browser tab or window, and the project says it can create callouts with different shapes, colors and styles. An example on the project page marks an Allow control in System Settings and adds a message asking the user to approve access.

The project describes its implementation as one Swift binary, with no background daemon, menu-bar icon, account or telemetry. It says drawing requires no macOS permission. The listed installation route is Homebrew; the page also documents building from source with Xcode 16 or later on macOS 14 or later. These are details supplied by the project, not independently verified findings.

Callouts can disappear automatically, be stopped with a command, or be tied to the process that created them, according to the documentation. The project also describes a copy-and-paste feature: text enclosed in a marker can appear on the sign as a copy button. Other listed options include spoken text and exporting an image for documentation or demonstrations.

A Visual Handoff to the User

AI agents can reach a point where they identify the next step but should not take it themselves. Examples include permission dialogs, two-factor codes, payments and signatures. A visual marker gives the agent a way to hand control to a person without relying on a terminal message that may be out of view.

The distinction between pointing and acting matters for both user control and clarity. Bigarrow presents itself as a guidance tool rather than an automation tool: the person can inspect the requested action and decide whether to proceed. The same approach could help with tutorials, support over screen sharing, or explaining unfamiliar software, though the project page does not provide independent usability studies or adoption data.

From Terminal Prompts to Screen Pointers

The project frames bigarrow around a limitation of agent workflows: an agent may be able to perform substantial work on a computer, yet still need a human to handle a protected or personally approved action. A plain-text request such as “click Allow” can be easy to miss when it appears in a terminal rather than beside the relevant control.

Bigarrow’s documented approach is to place a transparent overlay above windows and displays, including across macOS Spaces, then remove it after a set duration or when stopped. The examples on the project page show use in System Settings, Keynote, TextEdit, Chrome and Finder. They illustrate intended uses; they do not by themselves establish performance across all apps or Mac configurations.

“It only points.”

— Bigarrow project page

Testing and Compatibility Still Open

The supplied material is the project’s own documentation and examples. It does not include independent testing, user reviews, adoption figures or a comparison with other screen-guidance tools. The page says the demonstration uses macOS 27, but does not establish how every feature behaves across supported operating-system releases, display arrangements or apps.

It is also unclear how reliably the tool locates controls when labels change, are hidden, or are unavailable to the agent. The documentation offers coordinate-based pointing as another option, but coordinates can depend on window layout and display setup. The project says its overlays need no macOS permission; the scope of that statement and any limitations in particular environments are not independently confirmed by the supplied source.

Installation and Real-World Use

The next step for interested Mac users is to inspect the project documentation and try the listed Homebrew installation or build it from source, taking into account the stated system requirements. The project also provides commands for installing its Claude Code and Codex skills, which are intended to teach those agents how to use the utility.

Further evidence will depend on how the tool works in routine use: whether agents can target the right controls consistently, whether users find the handoff clearer than text instructions, and how it behaves across different apps and displays. The supplied report gives no roadmap or release schedule, so those developments remain unknown.

Key Questions

What is bigarrow?

Bigarrow is a macOS command-line utility and agent skill that draws visual callouts—such as arrows, boxes and labels—over application windows.

Can it click a button or enter text?

No. The project says bigarrow points but does not click, type or capture screenshots. A person remains responsible for the action.

Which AI tools does the project mention?

The project documents skills for Claude Code and Codex. The supplied material does not establish support for other agent products.

How is it installed?

The project lists a Homebrew installation and a source build using Xcode 16 or later on macOS 14 or later. Those requirements come from the project documentation.

Does it need screen recording permission?

The project says drawing requires no macOS permission and describes bigarrow as not capturing the screen. That claim is from the project and has not been independently verified in the supplied material.

Source: hn