We are biased to measure the things that are most easily measured, rather than the things that are most meaningful.
Many outputs are invisible or immeasurable. There is therefore always a gap between what can be measured and the actual contribution of any employee or individual. But if employees are incentivized to focus only on what is measured, it may mean that other responsibilities are neglected.
Organizations often measure inputs rather than outputs.
We may select metrics that are more consumable to non-experts in the name of transparency and accountability, rather than trying to measure what actually counts.
In some respects, the tables have turned: data is so abundant that organizations now look to find the right questions to ask to use their data, rather than looking for the right data to answer their questions.
See Abundant data makes us jump to quantitative data as the solution to every problem., especially Sam Ladner's comment on "data exhaust."
- Collecting analytics data is not the same as doing research
- Quant data is lossy
- Quantitative analysis is not inherently more reliable than qualitative data