The first time I ran this review properly, I ran it backwards, on the same two URLs from the rest of this cluster. I opened the scorecard, started ticking boxes, and punished a 900-word sibling that answered the question in the first screen, while our 2,400-word category roundup drifted past with a comfortable score because it looked expensive. Nobody had asked me what either page was for, and I had not thought to ask myself. That roundup is the one that settled at position fourteen. The 900-word page was the one I should have defended.
That is the failure a Content Effort audit exists to prevent, and the instrument comes from Google's Search Quality Evaluator Guidelines, the handbook more than 10,000 raters work from. Raters are told that effort may be direct human work or the systems that power a page, and that automatically translating thousands of pages without curation would not be considered to have effort. Search Engine Land's coverage of the 2025 rater update is the readable recap if the PDF defeats you, which it will around page ninety. What governs everything below is visibility. Effort has to sit in the Main Content, the rater term for the part of the page doing the job you arrived for rather than the navigation and the ads. If a stranger cannot see the work there, a Content Effort audit cannot score it, however hard the team worked.
TL;DR
- Classify page purpose before scoring anything. A short clip and a documentary were never meant to share one raw bar.
- Score six visible signals from 0 to 2 for a total out of 12, then band it in evidence language rather than an invented Google metric.
- Fix zeros with artifacts and first-party work, because "write more" is not a remediation.
- Calibrate reviewers with a high versus low pair pulled from the same query class.
- Library-level keep, refresh, or prune calls belong to a content quality audit. This instrument runs per URL.
How do you run a Content Effort audit?
Five steps, and the order matters more than the speed, because the order is where I got burned. Purpose comes first. Skip it and you will do what I did, punishing a short useful page and forgiving a long paraphrase, since length is the easiest thing in the world to see and usefulness is not.
Budget roughly fifteen minutes per URL for the first few. Mine ran closer to twenty-five, and most of the overrun was me arguing with myself about whether a screenshot of somebody else's dashboard counted as an artifact. It does not, for the record. After five or six URLs you land near five minutes, which is when a Content Effort audit stops being a workshop and becomes a habit.
- Name the page purpose. What job does this URL exist to do, and for whom? Write it as one sentence you would defend to the writer, out loud, without hedging.
- Use the Main Content. Read it, click it, watch it. Raters are told to spend real time inside the Main Content before judging it, and your reviewers owe the page the same courtesy, especially one they already disliked.
- Collect visible evidence only. Facts, artifacts, synthesis, craft, curation, and added value. Time logs do not count, and neither does "we used AI and then a human edited it," because neither is visible on the page.
- Score the six signals. Zero to two each, with one proof phrase beside every number, so the score is disputable rather than decorative.
- Verdict, then adds. Band the total, then list the cheapest evidence that would lift the weakest zeros. Adjectives are not adds.
What you want at the end is a table an SEO lead can hand to a writer with no translation layer in between. If the writer reads it and still cannot see what to produce on Monday, the review measured taste, and taste is not something you can assign. One layout choice helps more than it should: put the proof column left of the score column, so reviewers cannot record a number before they record why.
Full definitions live in what is Content Effort, and the wider system map lives in the Content Effort guide. This page is only the audit, because I have watched the definition essay and the scoring instrument eat each other inside one document more than once.
The six visible signals
Six editorial signals carry the score. They are not Google's weights and are not claimed to be. Cyrus Shepard is the named practitioner check here, and his framing is the one I keep returning to: effort is evidence of useful work left behind on the finished page. Residue, in other words. The rubric needs a name so teams stop quietly rebuilding it every quarter with slightly different columns, so call it the Visible effort evidence scorecard and train reviewers to fill the proof column before choosing a number. A score with no quote attached is an opinion wearing a number, and opinions are what turn a Content Effort audit into a meeting nobody scheduled.
Write one proof phrase per row. If you cannot write it, the score is a zero, even when the prose reads expensively, and that rule will feel harsh the first time it costs a page you like.
| Signal | 0 | 1 | 2 | Proof to capture |
|---|---|---|---|---|
| First-party information | Every fact exists elsewhere | Some first-hand color | Claims only this creator could make | Quote the unique claim |
| Non-replicable artifacts | Stock, scraped, or none | A few unique assets | Data, dated screenshots, methods, tools | File names and dates |
| Model-changing synthesis | SERP paraphrase | Useful organization | Reader's model of the topic changes | The decision rule that is new |
| Craft vs purpose | Format fails the job | Adequate | Skill matched to page type | Why this format is the right one |
| Curation fingerprints | No choices visible | Some structure | Omissions, point of view, sequence | What was left out on purpose |
| Added value vs substitutes | Interchangeable | Small unique slice | Does a job the rest of the SERP does not | Named substitute URLs |
Read those as six independent questions rather than one ladder, because a page can earn a two on craft and a zero on first-party information, and that combination has a name: a beautifully designed rewrite of somebody else's research. Score only what is on the page today. A 2019 screenshot attached to a claim about a 2026 interface is not a two, it is a liability. When two reviewers land more than two points apart on one signal, they did not capture the same proof, so re-run that row rather than the whole Content Effort audit.
How should teams measure Content Effort?
Sum the six signals, apply purpose in words, then choose adds. It is deliberately not a crawl metric, which disappoints anyone who wanted a number they could chart. The total exists to force a conversation about evidence, so it is close to worthless without the proof column beside it. Say a score out loud with its weakest signal attached and you will hear within five seconds whether the review was real.
Keep three claim classes apart while you do it, because this is where credibility gets spent. The rater guidelines language is documented and quotable. The contentEffort attribute described in Searchable's analysis of the Content Warehouse leak is reported by third parties, not confirmed by Google. Whether either carries ranking weight today is inference, and labelling it as such is your job, not the stakeholder's. A Content Effort audit stays useful precisely because it never needs that third class to be true.
One more warning, from a board deck I would like back. Do not average scores across a domain and call the mean a site-wide verdict. A homepage, a changelog, and a migration tutorial are three different bars, so blending them produces a number nobody can act on. Sample by template, score individually, then look for shared skeletons among the low scores. That pattern is the finding worth escalating.
Content Effort audit score bands
| Total | Evidence band | How to talk about it internally | Typical next move |
|---|---|---|---|
| 0–3 | Lowest-risk evidence | Recycled Main Content, little added value | Redesign the evidence, or noindex and merge |
| 4–6 | Low evidence | Some work, still substitutable | Two concrete artifacts before any expansion |
| 7–9 | High evidence | Satisfies purpose with visible work | Fill the remaining zeros without padding |
| 10–12 | Highest evidence | Hard to replicate from the SERP | Maintain, and use as the calibration example |
The bands are language, not scoring weights, and they exist so reviewers describe evidence instead of quietly guessing at rankings. Never publish "Google Content Effort: 8/12" on a client slide as though a crawler produced it, because the first technically literate person in the room will ask where the number came from. Publish the whole sentence instead: visible effort evidence is 8 of 12, the weakest signal is artifacts, and the fix is a dated screenshot of the workflow plus the export it produces. A band with no add behind it is theater with a spreadsheet.
Page purpose modifiers
The rater handbook already says the expected bar differs by page type, so apply the modifier in plain language rather than burying it in a multiplier nobody remembers in March:
- User-generated and short social content: participation and depth can supply the effort. Do not demand a white paper from a forum answer.
- Tools and calculators: the system is the effort, provided it works and is not a thin wrapper around a public API response.
- Reviews and "best" lists: first-hand use is the default two on first-party information, and unedited affiliate copy is a zero.
- Health, finance, and other consequential topics: effort without accuracy is not a pass, so loop in an expert before anyone celebrates a ten.
- Programmatic pages: unique database rows are not automatically unique effort. When the template is the Main Content, score the template.
That last line is the unpopular one and also the one that saves the most money, because a template scored once tells you about ten thousand URLs.
What does a high-effort page look like vs a low-effort page?
Calibration only works when both examples answer the same question, so hold the query class steady. Mine is "how to migrate [SaaS A] to [SaaS B] without losing [object]," because that is where the gap between real work and confident paraphrase is impossible to hide. Two pages, same intent, similar length, wildly different residue.
Low-effort URL, scored 3 of 12. It restates both vendors' migration docs, swaps synonyms, and shows no sign that an environment was ever migrated by the person writing. Screenshots are lifted from marketing sites. There are no failure cases, no rollback, and no mention of what breaks at scale. First-party 0, artifacts 0, synthesis 1, craft 1, curation 0, added value 1.
High-effort URL, scored 10 of 12. The author migrated a 12,000-row object, logged the three API endpoints that silently truncated fields, dated the interface screenshots, published the rollback procedure, and named the step in the official vendor docs that is simply wrong. First-party 2, artifacts 2, synthesis 2, craft 1, curation 1, added value 2.
Word counts can match almost exactly, which is the point of showing the two together. The residue cannot be faked, because the second page contains things that only exist if somebody did the work: a row count, three named endpoints, a dated screen, a tested rollback. Keep both URLs in a calibration folder so every Content Effort audit starts from the pair rather than from whoever has the strongest opinions that morning. New reviewers score both blind before touching live work, and disagreements get settled against these examples instead of against seniority, which is what makes the instrument survive a reorg.
Effort adds that actually move the score
Match the add to the zero. Unmatched length is how a low-effort page gets more expensive without getting better, and I have paid that bill. A Content Effort audit ending in "make it longer" has failed, and the writer will respond perfectly rationally by adding adjectives, because you gave them nothing else to add.
Walk the zeros in signal order, since the early ones unlock the later ones cheaply. First-party information and artifacts tend to produce added value as a side effect, almost for free. Synthesis without facts is a pretty outline, craft without purpose is a format flex, and curation without a point of view is a table of contents with ambition. Write every add as a noun somebody can be assigned: a screenshot, a number, a rule, an export. When a writer asks how many words, the review has not landed yet, so send them back to the zero and make them name the artifact first. It feels slower in the meeting and it is considerably faster in the CMS.
- First-party 0: run the workflow yourself, publish a number only you hold, or quote a primary document you actually retrieved rather than one you saw quoted.
- Artifacts 0: dated screenshots of your own interface path, a downloadable sheet, a working calculator, a methods appendix.
- Synthesis 0: a decision rule, a sequenced rubric, or a do-this-not-that the SERP does not already repeat. See content engineering for non-commodity pages.
- Craft 0: fix the format. A video script dumped into a blog template is a craft miss, not a keyword miss.
- Curation 0: cut the sections that exist only because a competitor has them. Deliberate omissions are evidence.
- Added value 0: name the substitute URLs in the working notes and state the job each one fails. If you cannot name that job, you do not have added value yet, you have a preference.
Origin is not an add, and this is where the room usually gets tense. The guidelines are clear that generative production alone does not determine the effort rating, so a Content Effort audit asks an AI-assisted draft the same six questions it asks anything else, and the remediation list comes back identical. The longer version of that argument lives in Content Effort and AI-generated content.
Does a Content Effort audit replace a content quality audit?
No, and merging them is how teams end up with neither. A content quality audit inventories a library and assigns keep, refresh, consolidate, or prune to every URL in it. A Content Effort audit scores visible labor on a specified set of URLs. They answer different questions and will regularly disagree, which is a feature. You can prune a high-effort page targeting a query you should never have chased, and refresh a low-effort page sitting on a query you have to win.
Sequence them by the question in front of you. Run quality first when the question is what should exist at all, and run the effort review when the question is whether this URL shows work. The product versions stay separate for the same reason: the content quality audit handles the library, and the Content Effort audit template handles the URL set. One escalation connects them. If more than a handful of sampled URLs share a skeleton and a sub-4 total, that is not a page problem, it is scaled low-effort production, and it belongs in a different conversation.
Here is the part I find hardest to say without sounding like I am selling. Checklists die in documents. I have written this rubric into four different Notion pages at three companies and watched every one drift within two quarters, usually about six weeks after the person who set the bar changed teams. The version that survives is a skill with stable scoring rules, run as a workflow, with an agent fetching the URL and a human doing the one job a human is irreplaceable for, which is disputing the zeros.
Metaflow is built for that loop, and the Content Effort audit template returns a per-URL table, evidence language aligned to the rater guidelines, a missing-evidence list, and prioritized adds, so an SEO lead is not rewriting the standard every sprint. Encoding operator judgment rather than chatting it away each time keeps discovery and execution in one place, so what is missing and what to add arrive together with the dispute trail intact. Metaflow holds the calibration pair, the rubric, and the run history in shared context, which stops the bar drifting and lets reviews compound instead of resetting.
Frequently Asked Questions
How do you audit Content Effort?
Name the page purpose, use the Main Content, collect only visible evidence, score six signals from 0 to 2, then prescribe adds that match the zeros. Do not start with word count and do not accept time logs as proof, since neither is visible to the reader. Metaflow's Content Effort audit template produces that table by default.
How should teams measure Content Effort?
Use the 0 to 12 visible effort evidence total, band it into 0, 3, 4, 6, 7, 9, and 10, 12, and write the page purpose modifier in plain language next to the score. Do not present the number as Google's own metric, because it is an internal instrument built from rater language rather than anything Google publishes. Metaflow outputs the same table on every run so the instrument stays consistent as reviewers change.
What does a high-effort page look like?
A page whose unique claims, artifacts, and decision rules cannot be reconstructed from the current SERP. Craft matches the purpose, curation shows up in what was deliberately omitted, and added value does a job the substitutes do not. Length is incidental. Keep your highest-scoring URL as the calibration example so the bar does not quietly drift downward.
What does a low-effort page look like?
Recycled or paraphrased Main Content with little originality and little added value: template swaps, vendor-copy listicles, and summaries of whatever already ranks. The 2025 rater guidelines are explicit that crediting your sources does not rescue a Lowest rating when almost all of the Main Content is recycled from elsewhere.
Does a Content Effort audit replace a content quality audit?
No. A quality audit decides whether a URL should exist in the library at all, while a Content Effort audit decides whether the URL shows visible work. Run both, in that order. In Metaflow they are deliberately separate templates so the two jobs do not collapse into one vague content audit that returns neither verdicts nor artifacts.
Sources
- Google Search Quality Evaluator Guidelines (PDF)
- Search Engine Land: Google quality raters and AI-generated content
- Cyrus Shepard / Zyppy Signal: Content Effort
- Google Search Central: Creating helpful, reliable, people-first content
- Searchable: contentEffort leak analysis
- Hobo: What is Google's Content Effort signal?
- Google Search Central: Spam policies
- Google Search Central: E-A-T gets an extra E for Experience
