Written by Kiwi Dynamics Team 1What exact workflow will this fix first
Any agency worth hiring should be able to name the single workflow they will fix first, conflict-check triage, correspondence drafting, time recording, whatever it is, not a vague promise to transform the firm. If the answer is broad and covers everything at once, that is usually a sign nothing specific has actually been thought through. A real answer sounds like one workflow, one measurable outcome, one timeline.
2Does a lawyer review every output
This is close to a non-negotiable for a law firm: nothing AI drafts, whether it is correspondence, a document, or a status update, should go to a client without a lawyer reviewing it first. An agency that is vague about this, or suggests the AI can send things unsupervised to move faster, is a red flag regardless of how polished the rest of the pitch is. The answer should be a clear, specific description of where the human checkpoint sits in the workflow.
3How is client confidentiality protected
Client confidentiality and privilege are not abstract concerns in a law firm, they are the whole basis of the relationship with the client. Ask exactly where data is processed, whether it trains any shared model, and what security measures are actually in place, not just claimed. An agency that cannot answer this precisely and confidently should not be handling a firm's matter data.
4Does it work with our existing practice management system
AI that requires a fee earner to work in a separate system and then copy information back into Actionstep, LEAP, or whatever the firm already runs is not saving time, it is adding a second job. Ask specifically how the tool reads from and writes back to the firm's existing systems. If the answer involves a lot of manual re-entry, the tool will not get used past the first month.
5What happens to our data after the engagement
Some agencies build a tool, hand it over, and are unclear about what happens to the firm's data and access afterward, especially if the relationship ends. A firm should know upfront whether its matter data, templates, and precedents stay under its own control, and what the offboarding process actually looks like. This is worth getting in writing, not just asked verbally.
6How is success actually measured
Success in a law firm engagement should be measured in something concrete: hours given back per week, faster response times, fewer missed follow-ups, not a vague sense that things feel more efficient. An agency that cannot commit to a specific, trackable measure before starting is unlikely to have one after finishing either. Ask what gets measured, and how, before the project begins.
7Can we see this working for another firm
A short reference call or case study from another firm, ideally one of similar size and practice area, tells a firm more in ten minutes than another hour of pitch deck. If an agency cannot point to a real example of a workflow actually running in production, that is worth noting, because building slideware is a very different skill from shipping something a firm relies on daily. Ask to see it working, not just described.
8How long until this is in production
Some agencies quote a build timeline in months for something that should reasonably ship in weeks, once scoped tightly to a single workflow. A long timeline is not automatically a red flag, but it is worth asking why, and whether that time is actually needed or just how the agency structures every engagement regardless of scope. A good answer explains the specific reason, not a generic project plan.
9What does it cost against the hours it saves
The right way to evaluate cost is against the hours it actually gives back, not against a competitor's price list. Ask the agency to estimate, honestly, how many hours a week the workflow currently costs the firm, and hold the proposed solution to that number once it is live. If an agency will not make that comparison concrete, it is worth asking why.
10Who fixes it when something breaks
AI tools need maintenance, models change, templates get outdated, integrations break when the practice management system updates. Ask who is responsible for keeping it working six months in, and what that support actually costs and covers.
Kiwi Dynamics builds production AI for NZ and AU firms exactly around these questions, one well-scoped workflow, always lawyer-reviewed, shipped and measured rather than pitched and left.