How do you decide what the success metric for a new feature should be?

Start from the behavior change you expect, not from what is easy to log. Name the specific user action the feature should make more common, choose a metric that only moves if that action happens, and pair it with a guardrail metric that catches damage elsewhere. Then write down the number that would make you call it a failure, before launch rather than after.

I start from the behavior I expect to change, and I write the metric down before we build, because after launch everyone becomes very creative about what counts as success. So for a feature meant to get people importing data on day one, the metric is the share of new accounts that complete an import in their first session, not the number of imports, because the number of imports goes up if our heaviest users import twice. Then I add a guardrail. On that one it was support ticket volume, since a bad importer generates tickets even while the completion number looks fine. And I write the failure line explicitly: below a certain number after four weeks and we roll it back rather than iterate on it. Having that written down is the whole trick, because it turns the launch review into a decision instead of a debate about whether ten percent was good or not.