Dr Jack with Beryl and Wayne
What data can be used to evaluate a project’s value? It’s a commonly-asked question and one that world-leading ROI (Return on Investment) expert, Dr Jack Phillips, covered in his recent NZ ROI Certification workshop.
We recently had the pleasure and privilege of hosting Jack here in New Zealand. While here, he ran an ROI Certification workshop and spent some time with the Complete Learning Solutions team.
When showing people how to evaluate projects and programmes we typically find some common questions get asked. Jack covered these in his ROI workshop, including one of the most common questions of all – what data is needed when evaluating a project’s value and where can it be found?
Calculating any project’s return on investment is easy if you collect the right data. However, finding the data to prove the project’s impact (Level 4) appears to be the stumbling block for most people.
Six levels of data used to evaluate a project’s value
The Phillips ROI Methodology uses the following six levels of data to evaluate a project’s value:
- Level 0: Input, where the measurement focus is on input into programmes. This includes indicators representing scope, volumes, times, costs, and efficiencies.
- Level 1: Reaction and planned action, where the measurement focus is on reaction to the programmes. This includes participants’ perceived value and planned action to make them successful.
- Level 2: Learning, where the measurement focus is on knowledge and skills gained. Evaluated factors include learning how to develop concepts and how to use skills and competencies to drive programme success.
- Level 3: Application and implementation, where the measurement focus is on the application and use of knowledge, skills, and competencies. It includes progress made and implementation success.
- Level 4: Impact, where the measure focus is on the impact of implementing programmes and processes. These are expressed as improvement in business measures directly linked to the programme or project.
- Level 5: Evaluating a project’s true value. The focus is on comparing the programme or project’s monetary benefits to its costs.
Having credible data is critical when proving a project or programme’s impact or ROI. Without relevant data, it is impossible to compare the before and after, or pre- and post-programme results.
Remember, one of the ROI Methodology guiding principles is that if no improvement data are available for a population or from a specific source, one must assume that little or improvement has occurred.
The types of data used to evaluate a project’s value
Here are some examples of the types of data used to evaluate a project’s value, and examples of measures required for each level of data analysis:
- Level 0: Types of programmes, number of programmes, number of people involved, hours of involvement, costs.
- Level 1: Relevance, importance, usefulness, appropriateness, intent to use, motivational, recommend to others.
- Level 2: Skills, learning, knowledge, capacity, competencies, confidence, contacts.
- Level 3: Behaviours used, extent of use, task completion, frequency of use, actions completed, success with use, barriers to use, enablers to use, engagement.
- Level 4: Graduation rates, infant mortality, crime rates, productivity, revenue, quality, jobs created, efficiency, incidents of disease, retention, customer satisfaction, etc.
- Level 5: benefit cost ratio (BCR), ROI (%), payback period.
How to know what is actually of value when evaluating a project’s value
It is important to measure what matters. Find out what is important to the organisation by looking at the measures associated with the organisation’s strategic or business plan(s). Ask yourself: If the project or programme was successful, what would be its impact on organisational results? Other good people to talk to are the organisation’s CFO, accountants, continuous improvement specialists, and/or quality specialists.
Planning for data collection must occur at the beginning of any project, not as an afterthought at the end. Plan for success upfront during the project’s analysis and design phases.
Want to know more about how to evaluate a project’s value? It’s easy! Simply visit our ROI webpage and also download our free e-book! The e-book – How to measure the financial ROI in your business initiative can be found towards the bottom of the Resources page.