Achievements at a glance:
- Enhanced transparency & communication of the SAI’s value and benefits for society
- Increased confidence of citizens in the role of the SAI
- Effective targeting of development efforts
- Eighteen regional SAIs have conducted SAI PMF assessments, four have done repeat assessments, and three have published their SAI PMF report
While SAI PMF has been used all over the world, OLACEFS stands out for its quick adoption and wide-spread use of the framework. Thanks to a strong regional appetite for organisational improvement, an engaged peer community, and robust donor support, the OLACEFS region has embraced and benefited from SAI PMF in an impressive way.
This is the story of SAI PMF in OLACEFS…
At the start of 2012, the OLACEFS community was facing a challenge. Supreme Audit Institutions from the region were struggling to find an assessment tool that adequately fit their needs. SAIs at the time were using peer review guides, internal tools, SWOT analyses and others generic assessment tools to monitor their activities, but none of these holistically covered their unique needs and circumstances, or incorporated INTOSAI standards.
There was broad agreement across the region that SAIs needed a tailor-made measurement tool to support evaluations within the framework of ISSAI 12, which covers communicating and promoting the value and benefits of SAIs. The OLACEFS community sought to address this by creating a new committee, CEDEIR, which was tasked with acquiring and disseminating knowledge on SAI performance assessment and strategic management. CEDEIR was also specifically tasked with identifying a useful assessment tool that would meet the precise needs of member SAIs.
While the task may have initially been daunting, CEDEIR was not alone in its quest to find a better assessment tool. This same need had been identified around the world, and the donor and INTOSAI community had recently come together to address this problem through the creation of a new tool – the Supreme Audit Institution Performance Measurement Framework, or SAI PMF.
The SAI is a tool that allows a SAI to assess itself against the International Standards for Supreme Audit Institutions (ISSAIs) and other international good practices. This flexible tool, which can be adapted for the SAI’s unique circumstances and for the specific type of assessment being carried out, can help the SAI identify areas for improvement within their organization.
Over the past decade, the INTOSAI-Donor Cooperation has supported the development and successful roll out of SAI PMF, and its adoption by the international SAI community is viewed as one of the major accomplishments of the Cooperation. The SAI PMF tool has proved beneficial not only to SAIs, but also to development partners who can use SAI PMF reports to realigning their aid efforts in support of a SAI’s greatest needs.
SAI PMF adoption – the benefits of peer support
The pilot phase of SAI PMF was launched not long after CEDEIR has launched its own search for a new assessment tool. The committee members were given an opportunity to review the methodology and flexibility of SAI PMF and found that the proposed tool met their criteria. The committee soon embraced the framework, and a new partnership between CEDEIR and the SAI PMF team in Oslo was born.
Right from the start, the OLACEFS region had shown a strong appetite for assessment. This was clearly on display at the first SAI PMF meeting in Lima. Held only six months after the SAI PMF pilot version was approved, this week-long session brought together SAI staff from across OLACEFS. This opportunity to learn about and discuss the SAI PMF tool face-to-face was a spark that energized many regional SAIs to get on board with SAI PMF. Early adopters such as SAI Brazil, Mexico and Costa Rica soon undertook pilot assessments, providing other SAIs in the region a chance to see the results and benefits of a SAI PMF assessment first-hand.
A strong sense of peer collaboration, along with a healthy dose of friendly competition, encouraged other SAIs in the region to emulate their colleagues and undertake SAI PMF assessments as well.
Regional champions were a major boon to the adoption and spread of SAI PMF in OLACEFS. From the very beginning, the new framework was backed by some of the largest SAIs in the region. CEDEIR members also exhibited constant willingness to collaborate with the SAI PMF team in Oslo to ensure the tool would work for SAIs in Latin America.
The region’s strong international focus and close cooperation with the wider INTOSAI community also meant that OLACEFS members were already attuned to the opportunities offered by a new evidence-based tool that followed international standards offered. By the time the first SAI PMF pilots were ready to roll out in 2013, several SAIs in OLACEFS were already eager to give the new tool a test run.
- Eighteen SAIs have engaged or decided to engage in SAI PMF assessments
- Four have done repeat assessments; allowing them to track changes over time.
- Three SAIs have published their SAI PMF reports; gaining with this national and international recognition for their efforts on transparency.
Support from development partners
Support for SAI PMF in OLACEFS was not only strong among the SAIs. Major development partners in the region were quick to identify the potential benefits of the new assessment tool as well.
The German Corporation for International Cooperation (GIZ) and the Inter-American Development Bank (IADB) have been consistent supporters of SAI PMF assessments in the region. IADB’s permanent backing of SAI PMF has been especially crucial for the success in the region. From funding the first pilot meetings to covering the logistics for peer support teams, IADB has helped SAI PMF gain a foothold in the region.
While promoting the use of SAI PMF, IADB also sought to ensure that the MoU Principle of “SAI-led initiatives” was followed. They supported all types of assessment, internal, external or peer assessment; the SAI PMF was first and foremost the SAI report. Important decisions, such as if to undertake the SAI PMF assessment, when and how to structure it, and the publication of the final report, were all entirely the purview of SAI leaders.
The use of SAI PMF has brought notable benefits to SAI-donor relations. The detailed information in a SAI PMF report has been used by development partners to improve their dialogue with SAIs. The reports have also shown support providers specific areas where an individual SAIs area preforming well, and areas where external support is most needed and would be most effective.
The value of a SAI PMF report in this regard can be seen in the case of SAI Colombia. The audit office’s initial loan request to IADB underwent revisions following the release of Colombia’s SAI PMF assessment. These changes helped to ensure that the final loan targeted clearly crucial needs at the SAI.
Country Highlights: SAI PMF impact in Costa Rica and Peru
The benefits that SAI PMF brought to the region can also been seen at the country level. In some cases, the results of a SAI PMF assessment have motivated changes within individual SAIs and have positively impacted the SAI and society at large.
The case of SAI Costa Rica
In Costa Rica, the SAI was able to use the results of the evaluation to make important improvements in different processes of the organization. The results of the evaluation highlighted the need to establish different action plans. One of them being the formulation of the Strategic Plan for building human capacity 2017-2022, with a guiding framework for human resources management. The evaluation showed that additional efforts were also needed to improve ISSAI implementation.
The SAI PMF report also helped SAI Costa Rica improve on previous efforts to share their value contribution to the public. For over ten years prior to the SAI PMF assessment, the audit office had released reports on the SAI’s “quantitative public value”, which helped citizens understand the SAI’s efforts to identify and report on cases of inefficient or incorrect management of funds. It also contributed to improve government transparency, generate savings, recoveries, increase in collection and other contributions of value and benefit to society.
Using the results from SAI PMF, SAI Costa Rica has been able to strengthen both its corporate and support processes and the audit process. The action plan for closing gaps is inserted in an aligned manner with institutional priorities; and this has contributed to strength the organization’s road map and its ability to meet the challenges and opportunities of the future.
The case of SAI Peru
SAI Peru has been a strong proponent of SAI PMF since its launch in OLACEFS. When SAI PMF was first introduced to the region, SAI Peru was chairing the CEDEIR committee that was searching for a new assessment tool. When the committee decided to begin promoting SAI PMF in OLACEFS, SAI Peru was at the forefront of strategy development to help promote at audit offices across the region on the uses and benefits of a SAI PMF assessment.
SAI Peru has also experienced positive change from a SAI PMF assessment first hand. The organisation’s 2014 pilot assessment highlighted several areas where the audit office was preforming well, and several areas for improvement. After reviewing and considering the results of the assessment internally, SAI Peru then took an additional step to publish the results.
This public release of Peru’s SAI PMF assessment became an opportunity for the SAI to engage with the national media. The SAI organized a press conference where it presented the report, highlighted both those areas where the SAI performed well and the areas where it could improve. This openness and effective stakeholder engagement helped to promote the SAI’s image as an institution that values and promotes of transparency, and also served to improve public awareness of SAI Peru’s value to citizens.