Versions per audience and testing method

When to split a script by audience

The question is not “should we translate”, but “does the context change”.

A translation makes the same script readable in another language. It is necessary as soon as you receive conversations in that language, and it changes neither the sequence, nor the tiers, nor the media.

A version per audience is something else: it changes the situations, the settings, the opening topics, sometimes the order of the tiers, because what works in one context does not work in another.

The decision criterion is simple. Split when at least two of the following three elements differ between two groups of fans: the time of day when they write, the topics they bring up spontaneously during qualification, and the tier at which they stop.

These three elements are read in your own data, not on a world map. That is why module 4 insists so much on measuring by cohort: without it, you split at random.

Two symmetrical mistakes to avoid. Multiplying versions without data produces a catalog of scripts that nobody maintains and that all age at the same time. Keeping a single version when the data shows two very different behaviors leaves money on the table and wears out the persona.

Versions over time

There is a second type of version, which has nothing to do with audience: the next version for the same fans.

The problem is concrete. A fan who has followed your script from start to finish has seen all the media. Sending them the same script again produces nothing. But rewriting a complete sequence every time is impossible to sustain.

The rule that works is economical: change only what has been seen.

The media already sent are replaced. Their descriptions are rewritten, because a description reused word for word on a new media is noticed immediately. The sequence, the conditions to move on and the prices, for their part, do not move.

Two important clarifications.

The description must match its own media. This is the classic version 2 mistake: you replace the images and keep the texts. The result is a conversation in which the description does not fit what the fan has in front of them.

A version is not an opportunity to revise everything. If you change the media, the prices and the sequence at the same time, you will never know what produced the difference. That is exactly the subject of the next section.

Testing without fooling yourself

Most “tests” run in this trade are not tests. Here are the four mistakes that invalidate them, and their correction.

Mistake 1: comparing two periods. “Last week we were at 28%, this week at 34%, so the new teaser works.” No: between the two weeks, acquisition changed, the mix of fans changed, the catalog changed. A test compares two groups over the same period, not two periods.

Mistake 2: changing several things. If version B has a new teaser and a different price, a positive result teaches you nothing usable. One variable at a time.

Mistake 3: reading too early. The first sends produce huge gaps that do not exist. Set in advance a minimum number of paid media sends before looking, and stick to it.

Mistake 4: not writing the stopping rule. If you decide after the fact what counts as a difference, you will always find what you are looking for. Write it beforehand: how many sends, and what gap in unlock ratio you will consider real.

What is worth testing

In order of yield observed in a normal operation.

The teaser of step 3. This is the point where the conversation tips toward the sale or not. A change of teaser produces clear, readable gaps.

The price of tier 2. It carries the volume, so even a small shift has a visible effect on revenue. Careful: track the unlock ratio and the revenue per conversation, otherwise a price cut will always look like an improvement.

The opening message. It applies to every arrival, so its effect spreads through the whole sequence. It is the fastest test to run since the volume is maximal.

The timing. The delay between the teaser and the tier. It is the least intuitive test and often the most profitable.

What is not worth testing at the start: tier 3, whose volume is too low to give a readable result, and wording variants of the same sentence, whose effect is drowned in the noise.

Document, or nothing accumulates

An operation that tests without documenting redoes the same tests every six months, often with contradictory conclusions.

A test register fits in one row per test: start date, end date, variable tested, version A, version B, number of sends on each side, unlock ratio on each side, decision taken. Nothing else.

This register has a second virtue, which becomes decisive in module 7: when you move to several creators, it will tell you what transfers and what does not. A result obtained on one persona does not automatically hold for another, and the only way to know is to have written down what you found, and on what.