Case study

The churn rate is the wrong number

A tutoring center wanted to know whether its paid search was worth running. The answer depended entirely on how you compute what a student is worth, and the standard method gets it wrong in a way that keeps bad campaigns alive.

The data on this page is artificial, for privacy. The analysis is the one I ran against a real center's records; the numbers have been regenerated from a model so that nothing identifiable is published. Code and full write-up: github.com/Wassaw-io/tutoring-center-analytics.

How long students stay

Two out of five students in this data had not left when the records were cut. Their tenure is unknown, only bounded below. Averaging those spells with completed ones is not a modelling choice, it is an arithmetic error, and it understates the median here by 35 percent.

The churn rate falls, and no student changes

The monthly hazard of leaving drops steeply across the window. Everyone reads that as students settling in, which implies you should fix onboarding.

But this data was generated with a constant hazard for every individual student. Nobody settles in. The curve falls because students differ in how likely they are to leave, the likeliest go first, and the survivors are a progressively more committed sample. The cohort gets more loyal without any student getting more loyal.

Formally: let each student carry a fixed multiplier drawn from a Gamma distribution with mean 1 and variance θ. Integrating that multiplier out is evaluating a Laplace transform, and it gives a population hazard of λ / (1 + θλt), which falls for any θ above zero. Drag θ to zero below and watch the curve straighten into the constant-hazard line.

The fitted θ is 0.48, and a likelihood ratio test against θ = 0 gives p = 1.6 × 10⁻⁹ once you account for the null sitting on the boundary of the parameter space. So the decline is real and it needs heterogeneity to explain it. What it does not establish is that any individual student's risk changed, and those two readings call for opposite interventions. Telling them apart needs a randomised holdout, not more analysis of this data.

What a student is worth

The formula on every marketing dashboard is revenue per student divided by the monthly churn rate. That is exact if and only if the hazard is constant, which the chart above just rejected. It also values gross revenue rather than contribution margin, and treats a dollar arriving in month 26 as worth a dollar arriving next month.

Doing it properly is one integral: margin times the area under the survival curve, discounted. Move the sliders and watch what it does to the decision.

What this changed

At the default settings the naive method values paid search at 3.09 dollars back per dollar spent, which clears the conventional floor. The survival-based number, discounted, is 2.70, which does not. Same students, same spend, same revenue. Only the arithmetic differs, and one of those two numbers keeps the campaign running.

There is a second effect that the acquisition report structurally cannot show. Paid-search students cost five to eleven times more to acquire and they churn 27 percent faster once acquired. Cost per acquisition is measured at the door. Retention is measured over the following two years. Nobody looking only at the first number will ever see the second.

The most useful finding was not a marketing variable at all. Attendance in the first eight weeks moves the hazard of leaving by roughly a factor of three across its range, and unlike every other variable in the model it describes a student who is still enrolled and still reachable by phone. Everything else here is a fact about students who are already gone.

Full derivations, the Cox model, the hand-rolled frailty maximum likelihood fit, and a demonstration that unobserved heterogeneity attenuates Cox coefficients toward zero are in RETENTION.md.