• back
  • IMPACT ASSESSMENT

  • Impact assessment
    Trade Support Institutions (TSIs) are increasingly under pressure from their stakeholders to demonstrate that their services provide significant and lasting positive changes for their clients. TSIs have traditionally focused on assessing their efficiency, but not the impact of their work. Quantitative measures are necessary and important, but they do not provide the information TSIs need to assess the benefits of their services for their clients. ITC provides advisory and training services to assist TSIs to assess the outcome and impact of their services.

    Information

    Organizations have been exploring new ways to measure results by shifting the focus from examining how a service is operated to considering the good that the service achieves. Evaluating the impact of a service is a crucial activity for all project managers and, furthermore, good performance measurement systems will assist managers to identify areas of the service offer that may need improvement. For more information and links to online resources related to impact assessment, please expand this section. 
     Donors too must measure the impact of the resources they provide to organisations, and their reporting needs are often different from each other. Many donor organisations prefer to use a logical framework as it allows for correlations to be made between activities, outputs and client satisfaction. Recently however, donors have been requesting that more information be provided concerning the positive changes that their resources instigate, so they can make better decisions on the allocation of those resources. In response to this TSIs are beginning to assess and report on the changes that are a direct result of the TSI’s service. The Canadian International Development Agency (CIDA) is one donor agency that provides tools to assist project managers to track and report on the changes in behaviour of their clients. Templates of these two tools can be found here.

    Data and Research

    In 2004 the World Bank pre-empted the upcoming shift in the way TSIs could assess results. The table below compares the key features of “implementation monitoring” against the new approach, “result monitoring”.
     

    ELEMENTS OF IMPLEMENTATION MONITORING 

    ELEMENTS OF RESULT MONITORING 

    Description of the problem or situation before the intervention Baseline data to describe the problem or situation before the intervention
    Benchmarks for activities and immediate outputs Indicators for outcomes
     
    Data collection on inputs, activities, and immediate outputs Data collection on outputs and how and whether they contribute toward achievement of outcomes
    Systematic reporting on provision of inputs More focus on perceptions of change among stakeholders
    Systematic reporting on production of results
     
    Systemic reporting with more qualitative and quantitative information on the progress toward outcomes
    Directly linked to a discrete intervention (or series of interventions)
     
    Done in conjunction with strategic partners
     
    Designed to provide information on administrative, implementation, and management issues as opposed to broader development effectiveness issues
     
    Captures information on success or failure of partnership strategy in achieving desired outcomes.
     

    Source: World Bank 2004 Report: “10 Steps to a Results-Based Monitoring and Evaluation System"

    In order to be able to demonstrate their “value for money”, more and more TSIs are beginning to measure the outcomes, the impact and effectiveness of their programmes and services, rather concentrating solely on reporting on their activities and consequent outputs.

    Advisory Services

    ITC provides consultation to TSIs interested in establishing an impact assessment framework for their organizations.
     The consultation focuses on:
    • Impact assessment approaches in the context of development
    • Establishing and implementing an impact assessment system
    • Key elements of the impact assessment framework
    • Analyzing and reporting the results.

    Training

    The TSI capacity building programme provides support to TSIs in their efforts to improve their own efficiency and effectiveness by equipping them with the knowledge and tools to measure their own results.
     These assessments allow them to:
    • Justify the financial support of governments and development partners by documenting the positive effects achieved with their resources. If TSIs are not in a position to demonstrate results, they will not win public support.
    • Improve the effectiveness of services by analyzing why they did not achieve the desired results. If TSIs cannot identify failures, they cannot correct and learn from them.
    • Adapt or improve the portfolio of TSIs’ services by focusing on those producing the greatest benefits for clients. If TSIs cannot identify success, they cannot reward it.
    • Retain or attract new customers by demonstrating the benefits derived from the use of specific services.

    The training provided under this module enhances the skills of TSIs to measure the outcomes and impact of their services.

    To do this, it provides information on how to:

    1. Formulate the main elements of an impact assessment framework – including the identification of relevant outcomes and impact measures
    2. Select the most relevant and efficient data collection methods
    3. Customize the impact assessment system to TSIs’ current portfolio of services

    Translate the results of the services into reports for governments and stakeholders. 

    Projects

    ITC is providing support under the PACT II programme to the African regional economic commissions of COMESA, ECOWAS and ECCAS to implement and monitor trade related technical assistance programmes: measuring the impact of the services is an integral part of the programme.
     ITC is also providing advisory services to Brazil’s Trade Promotion Organization, Apex-Brasil, to augment their current recording and reporting systems through assisting them with the development of relevant outcome and impact measures for their services.

    ITC will launch a comprehensive performance management training programme for TSIs in 2011. Impact assessment is one of the modules included in this programme. More details to follow.