Menu

CTR Experimentation

Measure the CTR of a group of pages before and after you change their title or description.

Updated

On this page

What it shows

A CTR experiment compares the CTR of a group of URLs before and after a change to their title, description, or both. It shows the raw change, a change adjusted for position, a comparison with your whole site, and a statistical test.

Kong Metrics does not change your website. You make the change on your website. The experiment measures the result.

When to use it

  • You changed titles or descriptions and want to know if the CTR changed.
  • You want to test a title pattern on some pages before you use it on all pages.

How to use it

  1. Make the change on your website. Start the test on the same day, because the test starts on the day that you create it.
  2. Open Optimize → CTR Experimentation, then start a New CTR Experiment.
  3. Define: type a Test Name and select the Change Type: Title, Description, or Both. Add URLs: Manual (one URL on each line) or Pattern (for example /blog/*). You can add up to 200 URLs.
  4. Validate: click Validate URLs. A URL is Ready when it has at least 7 days of data in the last 90 days. URLs with less data are not used.
  5. Review: check the list, then start the test.
  6. Wait for data. Then open the experiment to see the results.

How to read the result

On the result page, select a period: 7d, 14d, 30d, or 60d. The period sets the length of the before window. The after window starts on the test start date and ends on the last settled day, up to 60 days. A test completes automatically after 60 days.

CardMeaning
Δ CTRAfter CTR − Before CTR, in percentage points.
Δ Clicks, Δ ImpressionsThe change in clicks and impressions.
Δ PositionThe change in average position. A negative value is an improvement.
Position-Adjusted CTR ChangeΔ CTR minus the CTR change that the position change alone can explain, from the fixed CTR curve.
vs Site Average (DiD)Δ CTR of the test pages minus Δ CTR of the whole site. This shows if the test pages changed more than the site.
Confidence99%, 95%, or 90% when the test p-value is less than 0.01, 0.05, or 0.1. Otherwise n.s. (not significant).

The test is a two-proportion z-test on clicks and impressions. The result is significant when p is less than 0.05.

VerdictMeaning
Likely winnerCTR went up, and the test is significant.
Negative signalCTR went down, and the test is significant.
InconclusiveThe test is not significant. A 90% confidence result is also inconclusive.

URL Details shows the same values for each URL. PDF exports the result.

Example

All data is fictional. You changed the titles of 40 pages on example.com. Their average position stayed at 5.0.

  • Before CTR: 3.0%. After CTR: 3.6%. Δ CTR = +0.6 percentage points (a 20% relative change).
  • The position did not change, so the Position-Adjusted CTR Change is also +0.6 points.
  • The site CTR went from 2.0% to 2.1%. DiD = 0.6 − 0.1 = +0.5 points.
  • p = 0.003, so Confidence is 99% and the verdict is Likely winner.

The test pages improved more than the site. This supports the title change, but it does not prove it.

Limits

  • The result is an observation, not a controlled test. Season, Google updates, and other changes can also change CTR.
  • If stored data does not cover the test dates, the App gets the result directly from Search Console. Search Console can leave out some rows. Then the result page shows a Google Search Console estimate warning. Treat the change and the confidence as directional.
  • The position adjustment uses a fixed CTR curve. It is a model, not a measurement.
  • Percentage points and percent are different. A change from 3.0% to 3.6% is +0.6 points and +20%.

Choose which optional cookies we may use on kongmetrics.com and in the Kong Metrics App. You can change this at any time from "Cookie preferences" in the footer. Cookie Policy