Metrics – the risks in using number of enquiries / research requests in parliaments

Services still fall into the trap of promoting as a metric the number of enquiries/requests they get. Or their administrations impose it. Partly it seems an obvious measure, partly it is deceptively easy to collect and describe. It is, therefore very tempting to use it, especially if the figure is improving. Increasing numbers are an attractive message when a service is starting or has tried to increase interest from clients. But it is a potentially dangerous trap to use as a long term performance measure. There are at least three good reasons why services should not offer request numbers as a measure of success.

1. Request levels may fall for positive reasons, such as proactive briefings and better access to existing research products and published information. If the service is judged on the level of requests, it will incentivise resistance to delivering such improvements

2. Request levels may eventually rise to the real capacity of the service. Some research services are actively trying to manage the level of requests in order to remain viable and to answer satisfactorily those requests that really do justify a customised response. Capacity in a service is not easily managed as in most cases requests do not flow evenly but in peaks and troughs – and in a parliament demand cannot readily be shifted from peak to off-peak. The limit to capacity is not as high as a system with steady demand.

3. It is relatively easy to game a measure of enquiry numbers and such gaming would not be beneficial.

There have also been repeated reports that parliamentary library enquiries have become more challenging and complex since 2000 – simpler information needs are met now by self-service online sources, so only the more difficult information research tasks are outsourced to the library by Members or their staff. While libraries used to bulk their figures with many easy requests in the past, they must now work much harder to fulfil the same number of enquiries. If the focus is only on the headline figure, that change is not apparent and the explanation may be ignored. Complexity of enquiries is something that might be measured for an alternative metric but it would have been most useful if introduced in the early 2000s or before.

If you must use simple request numbers as a metric, better that it is carefully framed as a temporary measure suitable in a start-up phase but not appropriate to maintain long-term.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.