Citizens have a right to accurate information regarding what their tax money has been spent on. In addition, parliament has a responsibility to hold government departments to account for their performance, based on reliable evidence.
To enable this, the state has adopted “performance management” practices from the private sector. However, while the performance of a company can be measured and assessed based on straightforward financial metrics such as return on investment or net profit margin, assessing the performance of a government department is more complicated. It often involves assessing whether the department has achieved more complex outcomes and impacts that involve many role players, such as reducing poverty, increasing employment or reducing crime.
Government performance management is governed by the Framework for Managing Programme Performance Information (FMPPI), which was issued by the National Treasury in 2007. All departments are obliged to comply with the FMPPI when formulating their compulsory five-year strategic plans and annual performance plans. One of the aspects of performance management that the state has adopted from the private sector is a requirement in the FMPPI for performance indicators and targets to be specific, measurable, achievable, realistic and time-bound (Smart).
The auditor-general audits whether departments have complied with the FMPPI with regard to the performance information in their annual performance plans and related progress reports, and the audit results are included in departmental annual reports that are presented to parliament. In addition, departments are required to present quarterly reports on their performance to parliament, against the quarterly targets in their annual performance plans.
To meet the “realistic”, “time-bound” and “achievable” elements of Smart, performance indicators and related targets must be achievable within a year (the period of the annual performance plans) and must be achievable with a high level of certainty. That is, they cannot be ambitious. In addition, their achievement must be entirely within the control of the reporting department. A finding by the auditors that the performance indicators and targets of a department are not strictly Smart can result in a qualified audit report.
Unintended consequences
There are several unintended consequences of the use of Smart indicators and targets in annual performance plans to measure the performance of government departments. First, many of the strategic priorities of the government involve complex change processes with multiple contributing role players, the outcomes of which are uncertain and unpredictable, and not entirely within the control of the reporting department.
To avoid adverse audit findings, departments therefore often do not include performance indicators and targets related to such priorities in their annual performance plans, and there is no requirement in the FMPPI for them to do so.
Second, because the indicators and targets must be achievable, realistic and time-bound, departments usually avoid setting targets for indicators that are likely to take longer than a year to achieve. However, the achievement of outcomes and impacts related to complex socioeconomic problems often takes more than one year.
Third, while the system requires departments to set targets with a high level of certainty of achievement, political leaders and citizens expect departments to set ambitious targets.
Departments that try to set ambitious targets for indicators over which they do not have full control, such as “10% reduction in the murder rate” or “100 construction projects completed” often obtain adverse audit findings related to their performance information.
For example, departments do not have control over all the role players that have an influence on the progress of construction projects. There are also unforeseeable and unpredictable risks in construction, such as invisible underground rock making excavation difficult, unusually heavy rain delaying progress on site, or disruption by the local community.
Such risks can result in a project being incomplete by the end of the year, resulting in the target being “not achieved”. Setting an ambitious target for “number of construction projects completed” can therefore violate the “achievable”, “realistic” and “time-bound” criteria in Smart.
Setting an ambitious target for ‘number of construction projects completed’ can therefore violate the ‘achievable’, ‘realistic’ and ‘time-bound’ criteria in Smart.
The government performance management system therefore has the unintended consequence of incentivising departments to only choose indicators relating to short-term activities over which they have full control, and to only set targets with a high level of certainty of achievement. This in turn can result in annual performance plans that are not strategic.
When departments present their quarterly and annual reports to parliament the focus is often on the percentage of targets achieved. It is assumed that a department that achieves a high percentage of its targets is performing well, and vice versa.
However, this is not necessarily the case. For example, department A may have set Smart targets and indicators for short-term activities over which it has complete control and for which it has a high degree of certainty of achievement; may have achieved all of its targets; and may have obtained an unqualified audit report; but may not have made much progress in achieving the strategic objectives of government.
Conversely, department B may have set ambitious targets for indicators related to medium- or long-term outputs, outcomes or impacts over which it does not have complete control; may not have achieved many of them; and may have obtained a qualified audit report for not setting Smart indicators and targets; but may have done better than department A in terms of achieving the strategic objectives of the government.
So which department has performed better, department A or department B?
Beyond the ‘Smart’ criteria
There is therefore a need for the FMPPI to be reviewed with a view to developing a more nuanced performance assessment methodology for government departments, to address the unintended consequences of the system and enable the quarterly parliamentary performance review process to focus more on the achievement of strategic priorities.
• Dr Phillips is director-general, and Manyakanyaka chief director for planning & organisational performance, at the department of water & sanitation. They write in their personal capacities.
Would you like to comment on this article?
Sign up (it's quick and free) or sign in now.
Please read our Comment Policy before commenting.