Research Paper By Rachel McMahon
(Business Coach, AUSTRALIA)
Introduction
ICF-supported studies have indicated that almost all organizations state that they “know” coaching has been effective, but the evidence to support this is mostly anecdotal (Bechini, 2017).
In a similar study “the majority of participants indicated that they receive effectiveness feedback from the coaching client and that they never use ROI to evaluate their effectiveness. Thus it appears that “effectiveness” evaluations are mainly subjective, and not necessarily empirically valid measures of the coach’s actual effectiveness” (Gale et al, 2002).
This is surprising, especially given that in 2019 over USD$13 billion was spent on business coaching in the USA alone (IBISWorld, 2020).
I believe that with increasing scrutiny on business budgets, it is important to have a strong focus on outcomes and ultimately the business results that coaching is generating. I believe that to ensure the industry is credible and to attract and retain clients, coaches have a responsibility to demonstrate that coaching is a worthwhile investment.
This research paper explores three models and frameworks to measure the results of coaching. It then considers the essential elements that a business coach should include when engaging with coachees to build a results-focused coaching practice.
Ways to measure results in business coaching
The effectiveness of coaching can be measured at several different levels: personal feedback, behavioral changes, clients’ knowledge, and changes in business effectiveness and competitiveness (Fenyvesi et al, 2015).
Several different frameworks and models have been created to do this, including the ROI Framework Model, the Coaching Benefits Pyramid Model, and the Clockwork Tower Model. These models are outlined in this research paper.
Method 1: ROI Framework Model
The ROI Framework Model is one of the most well-known methods for measuring coaching results.
The first iteration of this method was developed in the 1970s by Kirkpatrick and was based on the premise that it is impossible to prove that a training intervention has been effective and that executives are generally satisfied with a causal link. Kirkpatrick identified four types of evidence: participant reactions, learning, changes in behavior, and results. (Kirkpatrick & Kirkpatrick, 2006).
Phillips added a fifth level to Kirkpatrick’s taxonomy, “Return on Investment” (ROI). Phillips believes that measuring ROI in business coaching is and will increasingly become imperative for organizations and coaching firms pursuing the highest standards of accountability. (Phillips & Phillips, 2004).
According to Phillips et al (2012), the five different levels of evaluation include:
Level 1: Reaction, Satisfaction and Planned Action
This level considers the coachee’s reactions to the coaching program, the perceived value, and the coachees planned actions.
Key questions to ask include:
- Was the coaching relevant to the coachee’s job and purpose of their roles?
- Was the coaching important to the coachee’s job and success?
- Did the coaching provide the coachee with new information?
- Does the coachee intend to use what they learned?
- Would the coachee recommend the program or process to others?
- Is there room for improvement with coach selection and match, coaching session duration, and frequency and the setting for coaching work?
Level 2: Learning
This level considers the extent to which coachees acquire new skills and the level of confidence they have to apply their newly acquired knowledge and insights on the job.
Key questions to ask include:
- Did the coachee gain the knowledge and insights identified at the start of the coaching?
- Does the coachee know how to apply what they learned?
- Is the coachee confident to apply what they learned?
Level 3: Application and Implementation
This level determines the extent to which coaches effectively apply their newly acquired knowledge and skills.
Key questions to ask include:
- How effectively are coachees applying what they learned?
- How frequently are coachees applying what they learned?
- If coachees are applying what they learned, what is supporting them?
- If coachees are not applying what they learned, why not?
Level 4: Business Impact
This level determines the extent to which the coachee’s application of acquired knowledge and skills has positively influenced the key measurements that were intended to improve as a result of the coaching experience. It is important at this stage to isolate the results from anything else that may have influenced the results.
Key questions to ask include:
- So what if the application is successful – what impact does it make to the business?
- To what extent does the application of knowledge and insights improve the business measure the coaching program was intended to improve?
- How does the coaching program affect output, quality, cost, time, customer satisfaction, employee satisfaction, and other measures?
- How do you know it was the coaching that improved these measures?
Level 5: Return on Investment
In this level, impact measures are converted to monetary values and compared to the program costs. For example, if improvements occur in productivity, to calculate ROI it is necessary to convert the measure of improvement into monetary value and then compare this to the costs of the program. Several steps need to be taken to evaluate and measure the value of coaching. These steps are summarised in the following diagram and can be divided into four phases: planning, data collection, data analysis, and reporting.
Key questions to ask include:
- Do the monetary benefits of the improvement in business impact measures outweigh the cost of the coaching initiative?
Method 2: The Coaching Benefits Pyramid Model
Leeman (2005) created the Coaching Benefits Pyramid. This model consists of four levels of evaluation and is similar to the ROI Model, however, it also includes “inner personal benefits” to demonstrate the importance of new skills and behavioral change.
The four levels include:
Foundation Factors
These factors include:
- The skills of the coach - such as listening, questioning, giving clear feedback, establishing rapport and providing support
- The personal attributes of the coach – such as knowledge, experience, qualification, being inspirational, having believed in the coachee’s potential
- The coaching process – including clear structure and discipline, being mentally challenging and stretching
- The coaching environment – providing a safe, supportive place to discuss confident and sensitive issues, providing time and space to think and reflect
The model suggests that when these four foundation factors are in place, the coachee is enabled to realize the level of inner personal benefits:
Inner Personal Benefits
This includes:
- Clarify and Focus – providing personal insights and exploration of themselves, their values and beliefs, providing a clear purpose and sense of direction
- Confidence – to believe in themselves, to feel more relaxed and less stressed, to raise their morale
- Motivated – to achieve, to improve both themselves and their organization, inspired to take action
Having realized the inner benefits for themselves, the coachee is mentally ready to produce outer personal benefits, which will be easily seen by others
Outer personal benefits
This includes:
- Enhanced skills, knowledge, and understanding – in both job-related skills and the ability to learn and develop themselves
- Improved behaviors – with individuals and teams in all forms of relationship
With these enhanced skills and/or improved behaviors, the coachee will be equipped and empowered to achieve the pinnacle of the pyramid.
Business Benefits
This includes:
- Business results such as improvements in performance, being more productive, enhanced career progression, resolution of specific problems or issues.
Method 3: The Clocktower Model of Executive Coaching Evaluation
The Clocktower Model of Executive Coaching Evaluation (Lawrence & Whyte, 2013)
is demonstrated as a broadly linear process. It commences with establishing purpose and ends with the measurement of financial ROI. (The ROI measurement only needs to be used if this outcome is outlined in the purpose). Alongside this process, is an ongoing (formative) approach to evaluation and measurement, which continues beyond the end of the coaching engagement.
The elements of the model include:
Purpose
The purpose is the pre-determined intent for the coaching engagement and is consistent with the business strategy. Establishing purpose is the first step in the process of measuring value and a key task is to ensure stakeholder alignment.
The Clocktower Model describes a broadly linear process, starting with establishing purpose and culminating with the measurement of financial ROI if applied. It will be argued that the measurement of financial ROI may or not be useful; what is important is the measurement of outcomes congruent with purpose. Alongside this process sits an ongoing (formative) approach to evaluation and measurement, which continues beyond the end of a coaching assignment, or series of assignments.
Motivation
Motivation refers to the commitment of the coachee to engage in coaching.
Methodology
The coaching methodology is the evaluation of the coaches coaching philosophy, background, experience, approach, and methodologies.
Behavior
Behavior refers to any aspects of the individual’s functioning that may be observed by others. For example, evaluation often considers changes in the coachee’s behavior. 360º Feedback is a common way to measure this.
Outcome
Outcome refers to the quantifiable results of coaching. The outcomes should be identified as specific coaching objectives at the commencement of the coaching engagement and should ultimately support the business’s strategic objectives.
Financial ROI
Financial ROI is defined as an explicit formula or calculation translating to the outcomes of coaching into a monetary value.
Formative Evaluation
Formative Evaluation is the process by which the purchasing client in an organization checks in with the process of coaching assignments.
Building results-focused coaching practice
Key Principles
In reviewing the evaluation methods, I have created several principles that I believe should be considered when building a results-focused coaching practice.
- Be clear on the purpose of coaching and the strategic goals that the business is trying to achieve.
- Use this information to develop a plan for how the coaching will be evaluated, and the required metrics to demonstrate this. Consider that there are multiple levels of success – from coachee knowledge, learning, and behavior to more concrete business results such as improvements in performance, being more productive, enhanced career progression, resolution of specific problems or issues, and ultimately return on investment.
- Before the coaching commences, use this information to create a “baseline” data to be collected and used as a point of comparison during and after the coaching.
- Continue the process of reflection and evaluation during the coaching engagement to remain on track to addressing the coaching purpose and achieving the desired goals.
- During and after the coaching engagement, where possible demonstrate progress against these metrics and share this with any relevant stakeholders.
Incorporating these principles into a coaching business
I believe that there are various ways that these principles can be incorporated into a business coaching practice.
- In the initial engagement and discovery calls, be clear in understanding the purpose of coaching, the coaching objectives and gaps that need to be addressed.
- Refine this understanding by setting clear goals and expectations in the documented coaching agreement. Agree with the coachee and any additional stakeholders upfront on what metrics will be used to track this. Also, be clear on the process and timeframes for how regularly this will be reviewed. Consider time frames, who’s responsibility is it to provide the data, and any time that may need to be built into the coaching engagement for reflections. Collect the initial baseline data that will be measured against this.
- Leverage technology (such as a coaching customer relationship management tool) to capture and record base-level data and metrics, set reminders to track progress at the agreed interview, and capture ongoing results.
Conclusion
I believe that measuring and tracking results in coaching is important.
I want to be known as a result focused coach. I want to have a clear understanding of the expected outcomes of coaching and would like to offer transparency in the results that the coaching is generating. I intend to use this as a way to differentiate myself as a coach from many of the other business coaches out there. To do this, I will integrate the key principles discussed in this paper into my business model.
References
Business Coaching in the US Market Size 2005-2026, 2018, accessed 18 June 2020
Bechini, 2017, “10 Easy Steps for proving the “value” of coaching”, International Coaching Federation, accessed 27 July 2020
Fenyvesi, Karpatine-Daroczi, and Vagany, “Measure the Success of Coaching”, accessed 27 July 2020
Gale, Liljenstrand, and Pardieu, 2002, “Coaching: Who, What, Where, When and How”, accessed 27 July 2020
Kirkpatrick & Kirkpatrick, 2006, “Evaluating Training Programs (Third edition)”, accessed 28 July 2020
Lawrence & Whyte, 2013, “Return on investment in executive coaching: A practical model for measuring ROI in organizations”, accessed 17 July 2020
Leedham, 2005, “The Coaching Scorecard: A holistic approach to evaluating the benefits of business coaching”, accessed 28 July 2020
Phillips & Phillips, 2004, “Measuring ROI in Business Coaching: An Overview”, ROI Institute, accessed 28 July 2020
Phillips & Phillips, 2015, “Measuring ROI in Executive Coaching”, ROI Institute, accessed 28 July 2020
Phillips, Phillips and Edwards, 2012, “Measuring the Success of Coaching”, ROI Institute, accessed 28 July 2020