What Is an Analytics Playbook? (Sometimes Called a Skill)

An analytics playbook is the recorded method for answering a recurring business question: the steps a senior analyst would take, written down so an AI agent runs the same rigorous investigation every time instead of improvising a new one. A lookup tells you what a number is. A playbook works out the harder things: whether growth is healthy, why a metric dropped, whether an experiment worked. Some teams call them "skills." The idea is the same: encode the expertise once, then run it on demand.

Key takeaways

• There are two kinds of data question. A lookup ("what was revenue last week") is one query. A higher-level analysis question ("why did revenue drop," "is growth healthy," "did the experiment work") needs a method, not a query. The playbook is the method.
• A playbook encodes what a senior analyst knows: which metrics to pull, how to slice them, what to rule out, and what a good answer looks like. The agent follows it instead of guessing a fresh approach each time.
• This is the line between a text-to-SQL chatbot and an AI analyst. A chatbot writes one query. A playbook-driven agent runs a complete, consistent investigation and shows its work.
• Playbooks ride on the semantic layer. The semantic layer fixes what each metric means; the playbook decides what to do with those metrics to answer the question.

---

What is an analytics playbook?

An analytics playbook is an encoded analytical method for a recurring business question. A playbook is to analysis what a recipe is to cooking: not the ingredients, but the method. The semantic layer holds your ingredients, the official definition of each metric and how your tables relate. A playbook is the steps you take with them. It is the procedure a senior analyst would follow to answer one recurring question, written down once so it can be run again and again.

Picture how a good analyst actually answers "why did sign-ups drop." They do not write one query. They break the metric into its parts, check each part, slice the one that moved by region and platform and cohort, rule out a tracking change before blaming the product, and only then say what happened and what to do. That sequence of moves is the expertise. A playbook is that sequence captured as a reusable method, so the agent runs it the same careful way every time, not a slightly different way depending on how the question was phrased.

You will also hear these called "skills." The word is different, the thing is the same: a packaged analytical method an agent can load and run. Sundial calls them playbooks because that is what they are, a play you run for a known situation.

Lookups vs. higher-level analysis: the two questions

Almost every data question is one of two types, and they need completely different tools. The first is a lookup: a fact you can fetch. The second is an analysis: a question you have to reason through. Most "AI analytics" tools only do the first and market it as if it were the second.

The valuable questions, the ones a business actually pays an analyst to answer, are almost all the second type. A few that come up at nearly every company:

• Is growth healthy? Not "what is our growth rate," but whether the growth you have will last: new versus resurrected versus retained, and whether newer cohorts are better or worse than older ones.
• Why did this metric drop? The decomposition-and-diagnosis question that turns one number into a cause.
• Did the experiment work, and should we ship it? Reading a test the right way: the effect, the guardrails, whether it holds up by segment, whether the early lift is just novelty.
• Is the marketplace supply-constrained or demand-constrained? A two-sided business grows differently depending on which side is short, and the fix is the opposite in each case.
• Which stage of the funnel should we focus on? Where the drop-off actually costs you the most, not just where it looks biggest.

Each of those is a methodology, not a metric. You cannot answer "is growth healthy" with a single SELECT statement, no matter how good the model writing the SQL is, because the answer is a chain of reasoning, not a value.

Why text-to-SQL can't answer the second kind

A query is a single decision. An analysis is a sequence of them. Text-to-SQL turns one sentence into one query and hands back the result. That is genuinely useful for a lookup. It falls apart on analysis, because there is no single query that diagnoses why revenue fell. There is a path: segment, compare, rule out, confirm.

Without an encoded method, even a capable agent improvises that path fresh every time. Ask it "why did churn rise" on Monday and it might segment by plan. Ask the same thing on Friday and it might forget to, or check a different cut, or skip ruling out a data glitch. The answers are inconsistent, and the inconsistency is invisible, because each one reads as confident and complete. A team cannot run on analysis that changes its mind based on how a question was worded.

A playbook removes the improvisation. It pins the method so the same question gets the same rigorous treatment whoever asks it. This is the key difference between a chatbot on your warehouse and agentic analytics: not that one is smarter, but that one follows a method you can trust and the other has to infer the method from the prompt each time.

What is inside a playbook?

A playbook is a small, structured spec for one analytical question. It is loaded only when a question matches it, and it lays out:

• The question it answers. "Diagnose a change in a top-line metric," "read out an A/B test," "assess retention."
• The metrics it needs. Pulled from the governed semantic layer, so the analysis is anchored in your official definitions rather than the model's guess at what a column means.
• The dimensions to slice by. Region, platform, plan, cohort, channel, whichever cuts the question actually turns on.
• The analysis logic. The steps, the branches, and the things to rule out before reaching a conclusion, like checking for a renamed event or a billing migration before blaming the product.
• The checks. Whether the data was fresh and complete enough to answer, and whether the conclusions actually follow from the numbers.
• The output. A structured answer a decision-maker can act on, not a wall of numbers.

The agent does not run every playbook on every question. It picks the one that matches what you asked, the same way an experienced analyst recognizes "ah, this is a retention question" and reaches for the retention method.

A worked example: "why did revenue drop?"

A revenue-drop playbook decomposes revenue into its drivers, finds the one that moved, segments that change to locate it, rules out a data artifact, and returns a diagnosis with a confidence signal. Here is the same question through a query generator and through a playbook, so the gap is concrete.

A text-to-SQL tool gives you: revenue last week = $X, down 12% week over week. Correct, and useless on its own. You still have to go figure out why.

The metric-change playbook gives you the why. It decomposes revenue into the parts that produce it (for many businesses, something like users times conversion times average order value), and checks which part moved. It finds conversion is the culprit, then slices conversion by segment and sees the drop is concentrated on one platform in one region. It checks whether anything in the data changed, a tracking update, a refund timing shift, before trusting the result. Then it lands the answer: revenue is down 12 percent, almost entirely from a conversion drop on Android in one market, and it lines up with a change that shipped that week. It says how confident it is, and it shows every step so a data person can check the logic.

A lookup tells you the number changed. A playbook tells you why it changed and what to do, the same rigorous way every time.

One produced a fact. The other produced a diagnosis a decision-maker can act on. The difference was not the model. It was the method the model was running.

Playbooks and the semantic layer: method vs. meaning

A playbook and a semantic layer do two different jobs, and you need both. The semantic layer is meaning: what "active user" means, how the tables join, which source is the truth. The playbook is method: what to do with those metrics to answer a question. Meaning without method gets you fast, correct lookups and no real analysis. Method without meaning gets you a confident investigation built on the wrong numbers.

The two are wired together on purpose. A playbook calls for its metrics from the semantic layer, so the analysis is built on governed definitions instead of improvised ones. That is what keeps a deep investigation from going wrong three steps in where no one would catch it: the foundation it stands on is the same trusted set of definitions everyone else uses. If you only read one companion piece, make it what a semantic layer is, because playbooks depend on it.

Who builds playbooks, and who runs them

Running a playbook and writing one are different jobs, split across two audiences. A business user just asks a question. The agent picks the matching playbook, runs it read-only, and returns the answer. The user never sees the machinery and cannot change any data by asking. That read-only default is what makes it safe to open analysis to non-technical people, the same split covered in trusting AI-generated SQL.

Data practitioners do more with the same agent. They author and extend the playbooks, encoding their own team's methods, and they model the tables and run data-quality checks to keep the context underneath honest. This is where the analyst role is heading. The job moves from reactively answering query requests toward architecting the context and the methods: defining the metrics, the relationships, and the playbooks, so that everyone's questions get a high-quality answer from a system the data team built. A team that runs this well likely needs fewer analysts doing manual pulls, and the ones who stay work on the harder problems.

How playbooks make analysis trustworthy

A method you can name is a method you can check. Four things follow from running analysis through playbooks instead of improvising it:

• Consistency. The same question gets the same rigorous treatment, no matter who asks or how they phrase it.
• It shows its work. A good agent tells you which playbook ran and what alternatives it ruled out, so a data person can audit the reasoning, not just the number.
• A confidence signal. The answer comes with how sure the system is, so a decision-maker knows a solid diagnosis from a rough read.
• Honest limits. When the data needed for a playbook is missing, the right answer is "I cannot answer this well yet, and here is exactly what is missing," not a confident guess. A clear "I do not know, here is why" beats a believable wrong answer, because a wrong answer delivered with confidence is the one that gets pasted into a board deck.

Common analytics playbooks

A small set of recurring patterns covers a large share of the routine analysis a company does. Sundial ships a library of 20+ horizontal playbooks, the recurring patterns that come up across most businesses, and teams add their own for the questions specific to how they operate. A sample of the horizontal set:

The point of keeping the core set small is the same reason it works: a tight library of trusted methods is something a team relies on, the way a tight set of fewer than 100 metrics covers most of what a business needs to watch.

When is a playbook the right tool, and when is it not?

Reach for a playbook-driven agent on the recurring analysis questions that today become a ticket and a two-day wait. Why a metric moved, whether growth is healthy, how an experiment read out, which funnel stage to fix. These have known methods, and encoding the method is exactly what makes the answer fast and consistent.

Be more careful in two places. For a genuinely novel, bespoke question that no established method fits yet, the agent can still investigate, but you are leaning on its general reasoning rather than an encoded method, so check the work more closely. And for governed reporting where every number has to tie out exactly, like a financial close, keep a human signing off. Use the agent to do the legwork, then have a person make the highest-stakes call. A playbook is also not a dashboard: for a fixed KPI everyone already watches, a dashboard is still the right surface, and the playbook is for the follow-up question the dashboard was not built to answer. Sundial gives you both in one place.

Frequently asked questions

Is an analytics playbook the same as a skill? Effectively yes. "Skill" is the term some tools use for a packaged analytical method an agent can load and run. Sundial calls them playbooks. The mechanism is the same: encode the method once, run it on demand.

What is an example of an analytics playbook? A metric-change diagnosis. When a number like revenue or sign-ups moves, the playbook decomposes it into its drivers, finds the one that changed, slices that change by segment to locate it, rules out a data artifact, and returns the likely cause with a confidence signal. Other common ones are retention and cohort, funnel conversion, growth accounting, and experiment readout.

What is the difference between a playbook and a semantic layer? A semantic layer is meaning, the official definition of each metric and how tables relate. A playbook is method, what to do with those metrics to answer a question. Lookups need the semantic layer. Analysis needs both.

How is a playbook different from a saved prompt or template? A prompt is free text the model interprets loosely. A playbook is a structured method tied to your governed metrics, with the steps, the checks, and the output it has to produce, and it leaves an audit trail. It runs a multi-step investigation rather than rephrasing one.

Does a playbook-driven agent replace data analysts? It does not remove the human, but it reshapes the job, and a team likely needs fewer analysts doing manual pulls. The work shifts toward authoring the playbooks and the context: encoding the methods and definitions so everyone's questions get good answers. Humans stay in the loop on judgment and the highest-stakes calls.

How many playbooks does a company need? Fewer than you would expect. A small set of recurring patterns covers most analytical work, which is why Sundial ships 20+ horizontal playbooks out of the box and teams add a handful of their own for the questions unique to their business.

If you want analysis that follows a defined method and shows its work, not a chatbot that improvises a new approach every time, that is what we build at Sundial.