Key Performance Indicators
Over time, grantee partners’ efforts to strengthen their organizational capacity yield improvements in CTOP’s Key Performance Indicators (KPIs), which we monitor on an annual basis to track progress toward our north star of improving positive, long-term outcomes for young people.
While all of CTOP’s KPIs are helpful in informing program improvement and are listed in full at the bottom of this page, the following KPIs best capture the social value grantees create:
The number of CTOP target population youth enrolled in core programming in active service slots. This KPI is critical as a proxy for the social value that grantees are creating in real time for CTOP's target population of young people. Program participants are only counted as occupying active service slots if they meet the program’s enrollment criteria and receive the appropriate kinds and levels of services needed to engender intended outcomes, as called for in the organization’s theory of change. In essence, this KPI tells us whether an organization is helping the young people it serves. You can learn more about the rationale behind the active service slot metric and how CTOP applies the concept through this report.
The number of CTOP target population youth enrolled in core programming who graduate from the program successfully. This KPI is defined in terms of participating in the full range and duration of core services and attaining the outcomes that the program promises. Young people who leave before completing the full scope of programming and/or who fail to attain the program’s outcomes clearly have not benefitted as intended and therefore are not included in this metric. Thus, like the previous KPI, this KPI tells us how well a program is helping the young people it serves – and gives us an indication of whether it may have helped them meaningfully.
The number of CTOP target population youth who graduate and, for the following six months, are actively engaged in education or employment. In the end, program participants who graduate a program successfully must go on to attain the long-term benefits (outcomes) it promises – or the program will not have much, if any, meaningful social value. This KPI, therefore, tells us whether a program actually has helped the young people it serves.
Results to Date
Beginning in 2020-21 and at the end of each subsequent grant year, CTOP will report the data for these three KPIs in the aggregate across the grantees in its portfolio. The number of active service slots our grantees were able to deliver in 2021-22 exceeded our initial expectations, jumping from 387 last year to 754 in 2021-22, and surpassing our projection made last year for 2022 by over 100. These increases are especially promising because they happened even while some grantee partners have not yet reached a stage of organizational development where their program participants would count towards these outcomes – and moreover, that they happened in the context of the Covid 19 pandemic, which exacerbated the very conditions necessitating these programs in the first place and substantially increased the challenge and complexity associated with the work our grantees do.
With regard to the second and third KPIs, we will note that most of the core youth-serving programs of our grantees are designed to require multiple years of participation. Therefore, the relatively small numbers for these two KPIs to date reflect in part the limited window of time that program participants have had to progress through and graduate from their programs, or to subsequently go on to achieve six-month post-graduating outcomes. In the years to come, we expect these numbers to increase steadily. And we also expect to extend the time for tracking long-term outcomes to one year initially but ultimately to at least two.
Key Performance Indicators
- Level of evidence for core program design, levels 0 – 4 *
- Level of confidence for core program effectiveness, levels 0 – 3 **
- Number of CTOP target population youth enrolled in core programming in active slots
- Number of CTOP target population youth receiving services but not in active slots
- Number of CTOP target population youth enrolled in core programming who leave unsuccessfully
- Number of CTOP target population youth enrolled in core programming who graduate
- Number of CTOP target population youth who graduate from core programming and for the following six months are actively enrolled in education leading to high school accreditation, post-secondary education/certification, or maintain unsubsidized employment ***
* 0 = program design by partner’s staff based solely on their ideas and experience; 1 = program design based on incorporation of widely shared practitioners’ “best practices”; 2 = program design based on applying fundamental research concepts (e.g., “attachment theory”); 3 = program design based on combining and incorporating elements that repeatedly are found in programs whose effectiveness has been proven by impact evaluation(s); 4 = program design based on implementing program in full that has been proven effective through impact evaluation(s).
** 0 = Asserted Effectiveness as supported by anecdotal data; 1 = Apparent Effectiveness as supported by internally collected outcome data; 2 = Demonstrated Effectiveness as supported by well benchmarked outcome data; 3 = Proven Effectiveness as supported by one or more rigorous impact evaluations.
*** In the case of an organization intentionally working to help its program participants transfer to another youth development organization, for those participants who are able to make the transition this will count as a success; however, for those who do not transition the long-term outcome as written applies.