Steps Premier League

A step counter competes with the phone in your pocket

Cohort analysis put a drop-off on the table. Not a feature request from anyone, but a shape in the data that said people were arriving and then not coming back.

The underlying problem is structural, and it is not specific to this app. Your phone already counts your steps. It tells you the number for free, on the lock screen, without anyone opening anything. A health app that shows you the same figure has to answer a question the operating system has already answered better.

1The phone counts steps whether or not the app is opened. The data arrives for free.

2A standing places you against people you know, so the number acquires a consequence.

3Falling behind is felt today; the health benefit is felt in a decade.

4Opening the app to check a position is the behaviour the league is actually buying.

↻A season ends, standings reset, and everyone gets a fresh reason to start

Three strategies were live options. Only one had no recurring bill.

Content, incentives, gamification. All three were real, all three had advocates, and there was appetite for roughly one.

Content and incentives buy attention you have to keep paying for. Another article, another voucher, next week, forever. Competition manufactures the reason to come back out of other users, which makes it the only one of the three whose cost is the mechanic rather than the fuel.

Content

Recurring

Every week needs new material, and the bill never stops.

Incentives

Recurring

Withdraw the reward and the behaviour it bought goes with it.

Competition

Shipped

One-time, then self-sustaining

Other users supply the reason to return. The cost is the mechanic, not the fuel.

Two design choices in what shipped answer part of that, and they are visible on the live league page rather than only in my account of it.

The first is the scoring ladder. Steps do not go onto a leaderboard as steps. They convert into cricket runs on four bands, one over per day, across a five-day season:

The bands are the mitigation. A raw step leaderboard ranks people by a continuous number, so everybody has a distinct position and somebody is always last by a visible margin. Bands collapse that: everyone between 10,001 and 15,000 steps scores the same four runs. Two people 4,000 steps apart are level on the table. It is a deliberately coarse instrument, and coarseness is what makes it survivable for the person at the bottom of a band.

The second is that the unit of competition is a team, not a person. Eight teams of seven, drawn along department lines. Being last in a squad of seven is a different experience from being eighth of eight on a public ladder: your bad week is absorbed rather than displayed.

Neither choice removes the cost I named above. A standings table still tells the last-placed team where it stands, every day, and there is an individual ranking alongside the team one. But both push in the right direction, and the honest version of this case study is that I can point at the shipped design rather than describe the design I would have wanted.

What shipped

Three retention strategies were evaluated. One shipped, and it shipped from nothing.

A recurring, time-boxed league built on step data the phone already collects: a five-day season, eight teams of seven, standings that reset when the season does. The design insight is in the restart: a league with no season end has no urgency, and one with no restart has no second chance for anybody who fell behind.

What happened, and the number that should have been here instead

Session time moved from 3.5 to 7.8 minutes. That was the declared north star for the launch, and it is the number I have.

It is the wrong north star for this feature. This was built to fix a drop-off, a drop-off is a retention problem, and the metric that answers a retention problem is whether people came back: a week-two-to-week-four return rate for the people who joined a league, measured against the same weeks for the people who did not. That is the headline this case study should have.

I deferred that instrument. The decision picked the metric, and I did not see the connection at the time.

Session time is a defensible supporting metric for a loop whose whole action is checking where you stand. It is a poor primary one for a health product, because a health product that works can shorten a session. Open it, log the thing, leave. Both are true at once, which is why the number needs an argument beside it rather than a bigger typeface.

The guardrails I would have watched

I did not have these. A retention mechanic that buys its numbers by burning notification permission is a loss wearing a win’s clothes, and none of the counter-metrics that would catch it were in the launch plan. They belong there rather than in a post-mortem, so here is the plan I would run it with now.

What I would do differently

I would instrument the cohort before picking the metric, not after. Everything above follows from that one sequencing error: the north star, the missing guardrails, and the fact that the strongest claim I could make about a retention feature is a number about session length.