Questions to ask before you hire an AI agency (and the answers that should worry you)

What should a good agency ask me before they quote?

Whether the role has written KPIs and written SOPs, and whether the work is mostly digital. When all three are true, an agent can usually take a real piece of that job. When they are not, there is no honest price yet, and a quote arriving anyway tells you the number came from a template rather than from your business.

This is the single strongest buy signal we know of, and it beats headcount and budget as a predictor. KPIs tell you what good output looks like, so the agent can be graded. SOPs tell you what the steps are, so the agent has something to follow. Take either away and the build turns into a discovery project wearing a build's price tag.

What happens when the SOPs do not exist is the more interesting question, and the answer tells you a lot. In our second-brain work the real answers were almost never in the wiki. They lived in Slack and WhatsApp threads, in two or three senior people's heads, and in documents that contradicted each other; ownership and versioning had to be settled before anything was worth connecting an AI to. An agency that has done this will say so. An agency that has not will offer to build anyway.

Other things a good one asks on that first call: which ten real examples from last month you can send over, who reviews the output and what a wrong answer costs, which system is the record of truth, who holds the credentials today, and what you would stop doing tomorrow if you could. In the buyer threads we read in August 2026 the most consistent complaint about consultants was the opposite pattern: they arrive already knowing your problems. If nobody has asked you a question you found slightly uncomfortable, you are being sold a product, not scoped.

No written SOP, no honest quote. The first invoice in that situation should be for writing the SOP down, not for building an agent on top of the gap.

Do I even need an agency at all?

Not if the work is one repeatable path that runs the same way every time and you have one motivated ops person. Build it in n8n, Make or Zapier and keep the money. That is the honest recommendation for a large share of the enquiries we get, and we make it on the call rather than after a paid discovery.

Our own rule of thumb on budget is published and unflattering to us: if your entire AI budget for the year is under $2,000, do not hire a consultant. A platform subscription and a few Friday afternoons will get you further than a small engagement, and a small engagement is the worst version of this work — big enough to cost real money, too small to change how anything runs.

An agency starts to earn its fee under four conditions. The work spans systems that do not have clean APIs. The exceptions are varied enough that judgment has to sit in the loop. Credentials and permissions have to be isolated per department. Or a previous build already failed and somebody has to work out why, which is a surprising share of what we do. Our build vs hire vs agency tool runs the same decision in a few minutes and will happily tell you to do it yourself.

The cheapest correct answer is usually a workflow you own. Ask the agency to argue you out of hiring them and see whether they can.

What does this cost, and what is the run cost after the build?

There are two lines on the bill: build and run. Practitioner-reported invoices put a simple single-purpose agent at $1,500–$5,000 to build with $300–$800 a month to run, and a production-grade build with written acceptance criteria at $8,000–$15,000. The run line is the one people forget, and it is the one that ends programmes.

The monthly bill has three parts: model and API usage, infrastructure, and maintenance. Maintenance runs $500–$1,500 a month with a solo operator or a boutique and $2,500–$8,000 with a full-service agency. Our AI agent cost guide has the full breakdown, including the market minimums the larger firms publish.

Here is the failure this question exists to prevent. A 20–50-person client gave every employee a personal always-on agent with its own token budget. Spend reached roughly $3,000–$5,000 a month and the programme was abandoned inside about two months. Two causes, both avoidable: idle agents burning tokens on heartbeats, polling, memory refresh and cron loops while nobody was asking them anything, and a flat rollout where everyone got one and few used one. What we would do now is shared departmental agents first, budgets per role, idle loops killed, and cheap tasks routed to cheap models.

So the question to ask is not "what does it cost to run", which invites a comfortable average. Ask what a bad month looked like on their last build, and what caused it. An agency that has run agents in production has a story and a number. One that has shipped demos will tell you it depends.

A proposal with a single number on it is a proposal missing the number that matters.

How do you stop the model bill running away?

Ask which model each step uses and why. Reaching for the largest model on every step is the most common expensive mistake we find on rescue work, and it is invisible until the invoice arrives. A good answer names the routing rule, the caching strategy, and a per-run token figure from a comparable build.

The spread is not subtle. On Anthropic's published pricing, checked 25 August 2026, Claude Haiku 4.5 costs $1 per million input tokens and $5 per million output tokens, while Claude Opus 5 costs $5 and $25 for the same volumes — five times the price per token for work that classification, extraction and routing steps do not need. Prompt caching charges a cache hit at 0.1x the base input price, which matters enormously when the same policy document or system prompt is resent on every call. Non-urgent batch work is discounted 50%.

Then ask about idle spend, because it is the part nobody demos. What runs on a schedule when nobody is asking anything? Is there a budget cap per agent or per role, and what happens when it is hit — does the agent stop, or does it keep going and you find out at month end? Is there a kill switch, and who can press it without calling the agency?

Model choice is an architecture decision with a price tag attached, and an agency that cannot explain its routing rule is going to bill you for the difference.

What happens after handoff, and who keeps it running?

Maintenance is where builds die. Agents break quietly: an API version changes, a form field is renamed, a rate limit trips, and nothing errors loudly enough for anyone to notice until the leads stop arriving. Ask who is contractually obliged to fix that, for how long, and how you will be told it happened.

The concrete version of the question has four parts. How long is the warranty window and what does it cover. Who is the named owner on their side and what is the response time. What alerting exists, and does it page a human or write to a log nobody reads. And what is included in the retainer versus billed separately, in writing, because "small changes" is where the relationship usually sours.

This was the single most repeated complaint in the buyer threads we read in August 2026: the build went fine and the handoff and maintenance were where things started to wobble. The one agency answer in those threads that got a warm reception led with a twelve-month warranty and a thirty-day refund window rather than a capability list. Buyers are not shopping for competence at this point. They are shopping for someone who will still be there in month four.

Ask who owns the agent after handover and what they are obliged to fix, before you sign the build.

Who owns the code, the credentials and the prompts?

You should, and it should already be true on handover day rather than promised for later. The repository sits in your organisation. The API keys sit on accounts you control and can rotate without asking anyone. The prompts are files you can open and read.

Prompts described as proprietary intellectual property deserve a hard look. They are text, you paid for them, and the claim exists to make leaving expensive. The same goes for an agent that only runs on the agency's infrastructure, a vector database in their account, or a set of connectors authorised against a founder's personal login that nobody documented.

There is a clean test, and we recommend running it before the final payment. Take the runbook, hand it to someone on your side who did not build the thing, and have them redeploy it from scratch. If that works, the handover was real. If it needs a call with the agency, the handover was a slide. The underlying question you are answering is simple: could you fire us on a Friday and still be running on Monday?

If leaving is expensive, you did not buy a system. You bought a dependency.

What will you refuse to build for me?

An agency with no refusal rules has no opinion, and an agency with no opinion will build whatever you ask for and invoice you for it. Ours are short enough to say on a call, and each one costs us revenue.

First: if the process is not repeatable yet, we do not build it. Run it manually, write down what actually happened ten times, and come back. Second: if it is repeatable and the path is fixed, we tell you to build it yourself in n8n, Make or Zapier rather than paying us to wrap it in something cleverer. Third: small, brand-new teams who have not settled how they work yet. Automating a process that is still being invented locks in a version of your company you are about to outgrow. Fourth: a rollout to everyone at once, which is the exact pattern behind the $3,000–$5,000 a month failure above.

The technical version of the same instinct is well documented. Anthropic's Building effective agents (19 December 2024) separates workflows, where "LLMs and tools are orchestrated through predefined code paths", from agents, where models "dynamically direct their own processes and tool usage", and recommends finding the simplest solution that works before adding autonomy. Most of what gets pitched to us as an agent project is a workflow in an agent costume. Our agents vs workflows vs RPA guide is the triage we run to tell them apart.

Ask for three things they have turned down this year. Vagueness here is the answer.

Who inside my company can see what the agent can see?

An agent inherits the permissions of whatever credentials you hand it, which means a badly scoped agent quietly becomes a way for anyone who can chat with it to read anything it can reach. Ask what identity the agent runs as, and whether that identity is per user, per department, or one shared account for the company.

We learned this the practical way. A shared assistant instance across a team turned into a shared-secrets problem: everything one person had connected was reachable by everyone else on the instance. The fix was isolation per user or per department with scoped credentials, and it is far cheaper to do at build time than to retrofit after somebody sees a salary spreadsheet.

The rest of the checklist: what gets logged and who can read the logs, what leaves your tenancy and to which model provider, what the retention setting is on that provider, and how an employee's access is revoked on their last day. In the knowledge-bot threads we read, the sharpest phrasing of the standard was to score permission leakage as an automatic fail — no other feature compensates for it. That is the right posture, and if your prospective agency has not raised the topic before you do, that is data.

One shared login for the whole company is a decision to have no permissions at all.

What does success look like at three months?

For a company-wide rollout, roughly 30% of staff as active users at three months is a win rather than a shortfall. The exec team goes first, the first department is onboarded two to four weeks later, and the months in between are spent building skills per department, choosing or building connectors, training people, and rearranging the agent hierarchy as usage shows what actually matters.

Adoption follows specificity. Nobody touched the generic assistant at one client until we built per-department skills; the tool did not change, the framing did. Any agency promising whole-company adoption in a month has either not run one or is counting logins.

Individual deployments move much faster, which is worth knowing when you are choosing what to start with. The best outcome we have had from a personal agent team was a 65-year-old lawyer in Minnesota, not technical at all, running three to five agents across intake and client comms, drafting and document review, and billing admin. He was self-sufficient in about two weeks and saves five to ten hours a week. He won because he already knew how to manage people: delegate, set expectations, review the work. Being technical was never the requirement. Knowing what good work looks like was.

So ask two numbers before signing: what they expect at three months, and what they would call a failure. An agency that will not name a failure condition has not agreed to be measured.

30% active at three months, exec team first. Anything shaped like "everyone will love it" is a sales number, not a plan.

Can I speak to a client who fired you, or one you turned away?

Ask for a build that went badly and what it cost the client. Ask for someone they told not to hire them. Hand-picked happy references tell you an agency can find three happy clients, which every agency can.

This matters more in this category than in most, because the public evidence is unusually poor. When we swept the buyer communities in August 2026 looking for people describing an AI agency they had actually hired, essentially every substantive answer came from agency owners rather than buyers; the question "has anyone actually hired one, what happened?" kept going unanswered. And roughly half of the page-one results for "best AI agent companies" are listicles published by agencies that rank themselves first. Treat that whole genre as advertising, because it is.

What is worth checking instead: third-party review profiles where the reviews are verified rather than self-submitted, platform partner status you can confirm at the source rather than a badge on their homepage — the n8n Expert Partners directory is the kind of list you can read yourself — and case studies with a number in them that names the company. Then read those case studies for shape, not size. A firm with impressive logos in an industry nothing like yours is a worse bet than a smaller one that has done your exact job twice.

The most useful reference is the unhappy one, and the willingness to hand it over is most of the signal.

Which answers are red flags, and which should reassure you?

Ten places where the answer is diagnostic. Read the fit-and-size row carefully, since it is the one where the correct choice is a firm larger than ours.

None of these is about technical skill, which is the part buyers usually try to assess and are least equipped to judge. They are about whether the agency has run something in production long enough to have been burned by it.

Ask every question in the left column on the first call. A good agency will enjoy three of them and dislike one, which is exactly what you want to see.

Where we fail this test ourselves

Run our own checklist against us and three things fail. We have no reviews on Clutch or any other third-party directory, so there is nowhere independent to verify us — everything you can read about our work is published by us, which is precisely the problem this guide warns about. Our published case studies (RiderDome, Emsa, Moovick, Samodrei and a private university) are software product builds for technology-shaped companies, not the internal agent rollouts at 20–50-person firms we now sell most, so they prove we can build rather than proving we can do your exact job. And we are small: eight people on the about page, with the founder writing these guides and running discovery calls, so the bus-factor question is fair and our only real answer is the ownership rule above — your repository, your keys, your prompts, a runbook you tested before the final payment.

Frequently asked questions

Sources

• Anthropic, model pricing (checked 25 August 2026) — the per-token spread between model tiers, the 0.1x cache hit multiplier, and the 50% batch discount quoted in the cost-control section.
• Anthropic, Building effective agents (19 December 2024) — the workflow/agent distinction and the argument for the simplest solution that works.
• n8n Expert Partners directory — an example of platform partner status you can verify at the source rather than from a badge on an agency's homepage.
• Cognio Labs, how much does an AI agent cost — build ranges, maintenance retainers and published market minimums, compiled from practitioner-reported invoices and firms' own directory profiles.
• Cognio Labs deployment notes, August 2026 — the token blowup, the rollout shape and the three-month adoption figure are our own client observations, anonymised. Community observations are paraphrased from public agent and small-business threads we read in August 2026; we have not linked individual threads because we did not capture permalinks during that sweep.

Related reading

• How much does an AI agent cost — the numbers behind the two-line bill, with market minimums.
• Build vs hire vs agency — a few questions that tell you whether to do it yourself, hire a developer, or bring in an agency.
• AI knowledge readiness audit — what to fix before any of this is worth quoting.
• AI agent development — how we scope, build and hand over, including what we refuse.