Risky business – performance indicators from a contractor’s perspective

Shane O’Regan - Associate Director, Driver Trett Qatar examines the practice of key performance indicators within the public works authority framework construction contract, for use on local roads and drainage projects in Qatar, and the financial risk allocated to contractors as a result.

Risk

Risk taking has become a notable area of concern for contractors in Qatar in recent years, due to demanding economic times and the intense competition within the construction industry. As with all projects, the construction contract is the focal point for the allocation of risks between the employer and the contractor. For that reason, it is always advisable that risks and responsibilities are clearly allocated within the contract. Although an employer and its consultants may decide the risk allocation policy for the project, both parties to the construction contract need to provide adequate risk management.

The concept of risk and the allocation of that risk within construction contracts, or indeed any contract, is not a new one. Being aware of the risks imposed by a specific contract and assessing those risks is vitally important. It will serve as a benefit to the project in terms of execution and delivery. The extent or level of success or failure in being aware of, and managing risks, is likely to have a huge financial impact on the parties involved; particularly the contractor. Therefore, it is in the contractor’s own self-interest to ensure that they are aware of the contractual risks that are being placed upon them.

Sir Michael Latham, author of the ‘Latham Report’, an influential joint government and construction industry report in the United Kingdom (also known as 'Constructing the Team') highlights that:

‘Risk is of course endemic in all forms of construction work. It can be managed, minimised, shared, transferred or accepted. It must not be ignored’¹ .

Box 1 - Calculating the KPI Amount

How to calculate the KPI adjustment?

Sub-clause 2.1.5 within schedule 5 of the framework contract states that the KPI amount is a 'fixed sum'. It is 5% of the work order price, excluding general Items.

It is important to highlight that if the contractor fails to achieve the maximum scoring, then this will result in a penalty to the contractor. Therefore, the KPI adjustment is not a bonus to the contractor, as it will give rise to a negative financial adjustment.

Another caveat that is worth highlighting:

Sub-clause 2.1.7 within schedule 5 of the framework contract states that: "the value or calculation of the KPI amount may be re-benchmarked if the engineer determines that circumstances have changed such that the originally calculated value is no longer valid".  However, it is not clear as to what criteria the engineer would use to determine a change in circumstances?  It appears that it is somewhat at the discretion of the engineer.

Contractor performance

In recent times, the Public Works Authority (PWA) in Qatar have chosen to use key performance indicators (KPIs) to define the level of performance expected from contractors when undertaking their work on site.

Whilst the application of KPIs is not widely used in Qatar (outside of public work contracts), it is important to highlight that other employers, such as private sector employers, may look to incorporate similar contract mechanisms on upcoming projects. It is often the case that, when a new contractual approach has been implemented by a certain sector or certain type of employer, other employers tend to follow suit. So, we may see more contracts in Qatar include KPI provisions over the coming months or years. 

From consulting with contractors engaged on the public work contracts, there is evidence to suggest that:

a) A high degree of uncertainty pertaining to the KPI risk exists;

b) This included uncertainty about the nature of the risk and the magnitude of the risk involved; 

c) Active risk management, on the part of the contractor, only became a focus during the construction stage; and

d) As a result, contractors were not fully aware of the potential negative financial implications associated with the KPI risk.

What is a Key Performance Indicator?

The Cambridge Dictionary online (2016) provides a general definition of a key performance indicator as, "a way of measuring a company's progress towards the goals it is trying to achieve". In other words, KPIs are a set of quantifiable measures that compare performance against key objectives.

KPIs, form part of the Public Works Authority’s Framework Construction Contracts for use on the Local Roads and Drainage Programme in Qatar. The framework contract states:

‘Key Performance Indicators' or 'KPIs' are the mechanism against which the Authority measures the contractor’s performance of the Works as described in Item 7 [Key Performance Indicators] of this Schedule B: Payment Schedules’.²

So at a glance, the word or phrase KPI does not set off alarm bells. Nor, in terms of risk, does the definition of a KPI appear onerous at the outset. But of course, as they say, the devil is in the detail. 

Taking a closer look, the primary objectives of the application of KPIs is contained within the opening paragraphs of schedule 5 of the contract. It states the inclusion of a KPI is to:

a) Incentivise contractors to implement a culture of health, safety, quality, stakeholder management, environmental culture and sustainability culture.

b) Measure the actual performance by the contractor under the Contract.

c) Promote a culture of continuous improvement by the contractor against a set measure of performance.

The framework contract also uses terms such as ‘Payment Linked KPI’ and ‘KPI payment mechanism’, thereby implying that the KPI results will have a direct financial impact on the contractor. Therefore, it is important to note that KPIs will not only be used to measure performance against a prescribed benchmark, but they will also influence the amount paid to the contractor.

The KPI results will, except in exceptional circumstances, have negative financial implications upon contractors (see boxes 1 and 3 below).

Risk management   

The importance of managing financial risks associated with, and arising from, construction projects cannot be under estimated. Risk management has been described as: “the art and science of identifying, analysing and responding to risk factors throughout the life of a project and in the best interests of its objectives”³. Ashley et al⁴ also suggests that there are three broad categories to risk management. These are:

• Risk identification.
• Risk analysis.
• Risk response.

In this instance, we first identify the risk; the inclusion of KPIs within the framework contract. In terms of risk analysis, one needs to consider both the probability of occurrence and the likely financial consequence. Once the risk analysis is complete, a risk response can be prepared. This may include, for example, the contractor electing to focus their efforts on achieving the best possible KPI scores by ensuring they have designated resources assigned to the task identified in Box 2 (below).

The example in Box 3 (below) is based on a relatively positive KPI score of 80% and a contract value of QAR 500,000,000. There are, of course, projects with higher contract values, and these projects may not secure a KPI score as high as 80%. This could potentially expose a contractor to deductions in the region of QAR 7,000,000 or higher.

 All of these possibilities mean that a contractor’s approach to risk management needs to be proactive, focused, and engaging. Ultimately, a contractor needs to consider to what extent they may include the cost of such risk within its tender bid, while at the same time, ensuring their tender bid is competitive enough to win the project.  During the construction stage, the contractor also needs to obtain the best possible KPI scores, and to achieve this, the contractor will need to ensure that they have assigned sufficient resources to achieve these goals.

In conclusion, the potential financial consequences associated with KPIs illustrates the importance of contractors ensuring effective risk management procedures, together with associated resources, are in place from tender stage through to construction completion of PWA projects.

Box 2 - Elements to the KPI Process

There are five key components to the ‘Payment Linked KPI’ process against which the contractor’s performance is measured. These are:

1. Health and safety.
2. Quality.
3. Programme completion.
4. Stakeholder management.
5. Sustainability and environment.

Each of these five elements also contain sub headings or, what are also known as, ‘KPI sub-sets’:

a) Health and safety has three sub-sets:

1. Assesses the number of significant injuries that occurred on the project.
2. Measures the number of road traffic accidents in relation to the works.
3. Under health and safety, assesses the number of near misses reported during the 12-month period.

b) Quality has also three KPI sub-sets:

1. Measures the number of non-conformance reports (NCRs) closed out during the period in question.
2. Assesses the failure rates of inspection and test results.
3. Assesses the number of defects identified at the time of handover.

c) Programme completion has only one sub-set:

1. Measures the level of compliance with the completion milestones as stated in the work order.

d) Stakeholder management has only one KPI sub-set:

1. Measures the extent of effective stakeholder management, and efficient coordination and interfacing with key project stakeholders, such as the local residents.

e) Sustainability and environment has only one KPI sub-set:

1. Relates to compliance with all statutory requirements set out by the Ministry of the Environment, and complying with the construction environmental management plan (CEMP) as required by Qatar construction specification (QCS).

Box 3 -  Example: examine the potential impact of the KPI risk on a project using a contract value of QAR 500,000,000 and a KPI score of 80%.

Table 1
Project A	'a'	'b'	'c' = ('a' - 'b')
Financial Particulars	Planned  Cash Flow of Works	Cash Flow for General Items	Planned Cash Flow Amount (Excluding General Items)
Planned Value - Year 1	260,000,000	39,000,000	221,000,000
Planned Value - Year 2	240,000,000	36,000,000	204,000,000
Work Order Price	500,000,000	75,000,000	425,000,000

 

Table 2
	KPI CALCULATION FOR YEAR 1 FOR PROJECT A	QAR
(a)	Planned Value of Work - Year 1	260,000,000
(b)	Deduct: Class A; General Items	39,000,000
(c)	Net Planned Value of Work = the Planned Value less the General Items; 'c' = 'a' - 'b'	221,000,000
(d)	Deduct the KPI Amount. Establish the KPI Amount by multiplying the Net Planned Value of Work by 5%; 'd' = 'c' x -5.0%	-11,050,000
(e)	Add the Incentivisation Amount. The Incentivisation Amount is established by multiplying the KPI Score '80%' x KPI Amount 'd'	8,840,000
(f)	KPI Adjustment for Year 1 is the KPI Amount plus the Incentivisation Amount; 'f' = 'd' + 'e'	-2,210,000

 

Table 3
	KPI Calculation for Project A	QAR
(a)	KPI deduction amount: Year 1	-2,210,000
(b)	KPI deduction amount: Year 2  (Also using 80% KPI scoring)	-2,040,000
(c)	Total KPI deduction for Year 1 and 2; ( c ) = ( a ) + ( b )	-4,250,000

Conclusion

• The KPI adjustment for year 1 gives a negative adjustment of QAR 2,210,000.
• If, for year 2, the same KPI scoring were to be applied, a total deduction to the Contract Price of approximately QAR 4,250,000 is made.
• This equates to almost a 1% deduction from the Work Order Price of QAR 500,000,000.

¹ Latham, M. (1995). Keynote Address by Sir Michael Latham. In M. Odams and J. Uff, Risk, Management and Procurement, London: Construction Law Press, pp. 1-1.

² Public Works Authority. (2017). Framework Construction Contract for Local Roads and Drainage Programme. Doha: Public Works Authority.

³ Wideman, RM., Risk Management, Project Management Journal, September 1986, pp 20-26.

⁴ Ashley, D.B., Dunlop J.R. & Parker, M.M, ‘Impact of Risk Allocation in Construction Contracts’, Risk, Management and Procurement in Construction: Centre of Construction Law and Management, King’s College London, 1995, pp. 113.