Choose product metrics that match a real task
Define measurement for a workshop-booking improvement by separating outcomes, events, denominators, failure signals, and research questions without fabricating performance claims or equating clicks with success.
Start with a question about the task
A metric is useful only when its meaning connects to the product's purpose. More clicks on a workshop page could indicate interest or confusion. Before choosing an event to count, state the task and the change you hope to understand. Measurement should complement direct research rather than turn every design question into a dashboard.
Define events and their denominators
Define the event, eligible population, time window, and exclusions. Distinguish a request from a confirmed booking and a page view from a person. Add signals for failure and recovery so the team can interpret apparent completion. Collect only data justified by the question, with appropriate privacy and operational review. Separate what the system knows from what the person has actually been told. A technically valid operation can still look unsuccessful when its confirmation is delayed or ambiguous. Describe the state before the action, while work is pending, and after it completes. Then name the information that remains available if the person leaves and returns. These notes become concrete requirements for the eventual implementation.
Plan booking-confirmation measurement
Plan measurement for a revised booking confirmation. Write definitions for submission attempted, booking confirmed, confirmation displayed, and support requested about status. Use fictional records to check that the definitions classify uncertain or repeated requests consistently. These are proposed definitions, not results from an experiment or evidence that the design improved. Work through the example once from the customer's perspective, once from operations, and once after an interruption. These passes examine different assumptions; none replaces research with people who use the finished service. Preserve the same task and content across passes so that changes reveal weaknesses in the design rather than unrelated differences in the scenario. Capture each weakness as a specific revision, not a general impression.
Investigate misleading changes in counts
A rising completion rate may hide a change in who starts the process. Duplicate events can inflate counts, while missing failure events can make a broken route invisible. If a measure encourages hiding a helpful cancellation option, it has become detached from the person's needs. Revisit the question and its interpretation. Distinguish a missing cue from a missing capability. Product content can explain an available recovery path, but it cannot create a recovery path the system lacks. Annotate where the interface needs a different state, a preserved value, or an operational response. Give each case a visible outcome. Otherwise an apparently helpful message may simply send someone around the same unsuccessful loop with more words to read.
Keep interpretation alongside definitions
Review the measurement plan with the people responsible for implementation and analysis. Ask what each possible change in a metric could mean and what additional evidence would distinguish those explanations. Keep definitions alongside any future report. The objective is an interpretable account of behavior that supports decisions, with clear limits on what the numbers can establish.
Worked decision
Two submit presses may belong to one booking attempt. Define the relationship before interpreting counts as two people or two confirmed places.
References & further reading
Examples and exercises in this guide are original teaching scenarios.
