Report unfolds ADB’s poor performance in Pakistan

ISLAMABAD: An evaluation report on ADB’s portfolio of $11.46 billion for Pakistan in the last decade, that is, from the year 2002 to 2012, reveals that the success rate of the operation has drastically declined from 55 percent to just 29 percent for overall loans owing to a variety of weaknesses.
This evaluation covers all 116 loans (including Multi-tranche financing facility [MFF] and grants approved from 2002 to 2012, for a value of $11.46 billion, of which 69% was non-concessional loan financing, 27% concessional loan financing funded by the Asian Development Fund [ADF]), and 4% grant financing obtained from other sources such as the Japan Fund for Poverty Reduction. The sovereign portfolio included 60 project loans, 33 program loans, and 13 grants. In addition, ADB approved for 10 private sector loans for $620 million. The evaluation report prepared by ADB’s internal independent department disclosed that, based on the ratings given in the project/program completion reports (PCRs), the performance of ADB-supported operations has been poor.
Of the 61 programs and projects, supported with 93 loans approved from the year 2002 to 2012, 43 have been completed, and PCRs are available for 34 of them. Internally Evaluation Department validated 18 of the PCRs. When validation ratings are considered, ten (29%) of the programs and projects are successful or better, eleven (32%) less than successful, and thirteen (38%) unsuccessful.
This compares with Pakistan’s success rate of around 55% during the earlier periods of lending to the country. Although most sector programs had low success rates, the completed emergency assistance has received a good rating. PCRs are still to be prepared for the significantly smaller portfolio of more recent projects.
Projects have been more successful than program loans, at 38% versus 15%. The Multi-tranche Financing Facility (MFF), now 70% of lending and 50% of all operations, have had one PCR, which was rated successful. In fact, some very important program loans that completed disbursements several years ago have no PCRs as yet. The absence of such essential self-assessments has made it more difficult to fully evaluate the program over the last 10 years.
This evaluation finds that the weak performance is partly due to the key changes that happened in the ADB portfolio after 2001, in terms of modality and sector focus. In particular, ADB supported a huge governance agenda through a large number of complex program loans in a difficult political context, and also various projects that were weak in design.
Significant course changes in the portfolio over the period may have had some effect as well. Program loans were widely adopted, but successor tranches or loans that could have helped to deepen the reform or capacity development were not released in many cases due to slow progress, and in some cases, request from the government. Policy-based lending often requires a series of consecutive loans for reforms to become firmly established.
When the IMF program went off track and ADB’s program lending was discontinued as per standing agreements, there was a limited effort to continue the reform agenda through other modalities such as the project modality. ADB’s support to PSM stalled. Shifts in strategy and perceptions, partly driven by crises in Pakistan or lack of support for program conditions, but also partly by ADB’s shift in sector priorities, were the causes of such stops and starts.
The report states that because of re-organisations in ADB, which moved the Pakistan program to a new department, it encountered numerous staff changes. Staffing in the resident mission was insufficient, particularly in the first half of the decade when there were many projects to administer. Although the provisions for staffing improved in the second half, several important staff positions were left unfilled for a considerable time.
Furthermore, the deteriorating security situation in both rural and urban areas restricted the involvement of international staff and consultants, and rendered the scheduling of field activities and supervision difficult. The security situation also made surveys more difficult. Lastly, there were two major natural disasters and one financial crisis over the period. Pakistan has always had lower than average project success rates, when compared to the historical ADB average (68%).
This is indicative of a difficult country context for project implementation, with problematic federal-provincial-local government coordination, stakeholder discord, unpredictable crises, and procurement, land acquisition, and safeguard implementation issues.
The largest sector of operations in the sovereign portfolio was public sector management (23% of total loan approvals). Emergency support after an earthquake and several floods ranked next at 16%, closely followed by energy operations and finance sector development at 14% each, with transport operations at 11%. Non-sovereign loans at 5% of the portfolio were extended mainly to power companies, also to investment funds and medium-sized enterprises.
Recommendations of ADB:
(i) Given the current situation in Pakistan in regard to human development, public services provision and governance, and the risks of natural disasters, ADB’s portfolio needs to include significant investments in pursuing a visible development impact on the poor, and reduce vulnerability to disasters. This is in line with ADB’s corporate agenda of inclusive and environmentally sustainable growth. Reforms and capacity development in several sectors remain highly necessary, but ADB should prepare any program lending or TA for these very carefully and with a long-term perspective, given the difficult experience of the recent past.
(ii) ADB must pursue structural reforms through sector-specific initiatives using a programmatic approach. Reforms in various sectors should not be lumped together into one large program. This was the case with many PSM loans during the review period. Budgetary support needs to be designed as a countercyclical support facility in times of crisis. The programmatic approach requires an extended timeframe and considerable staff resources. Policy conditionalities need to be carefully selected in dialogue with the government and key stakeholders through a transparent process that strengthens accountability. As the Operations Manual has consistently prescribed since 2003, implementation experience with past program loans needs to be carefully reviewed. ADB should also look into other modalities to pursue reforms in sectors given repeated cases where structural reform tranches were not pursued or cancelled, ADB should also look into other modalities to pursue reforms in sectors.
(iii) While cancellations of loans may be required if they are slow moving or the situation has changed, comprehensive spring cleaning of portfolios across the board may inadvertently lead to cutting the potential effectiveness of some project loans that are prematurely closed. Efforts to improve disbursement efficiency and instill more discipline in implementation should be accompanied by extensive consultation with clients at various levels and by careful assessment of the gains versus the losses (i.e., the costs of incomplete outcomes). Given the planning and implementation environment in Pakistan, long project startup and implementation periods should be taken as a given.
(iv) ADB’s energy strategy in Pakistan and its reform achievements need to be reviewed to determine whether a course change is required. Assuming that the current investment program will yield good results, ADB needs to take advantage of the political will demonstrated by the new government and intensify efforts to improve the power sector’s financial situation.
(v) As part of its infrastructure support, ADB has to consider expanding its work in urban and municipal services and social protection, given their direct effect on human, who are still poor in Pakistan.
(vi) ADB could help strengthen Pakistan’s disaster response capability by, for example, increasing its support for the National Disaster Management Authority.