Developing a Supplier Scorecard and Performance Management Process
Return to Newsletter home>>
A Vantage Partners Case Study
The Challenge
Lack of clarity around supplier performance (especially for suppliers of services related to clinical trials and other complex projects) made it impossible to identify the best suppliers with whom to create preferred relationships, to award specific projects to the most qualified supplier, or to work jointly with suppliers to improve performance.
Prior efforts over the course of a year failed to produce a viable scorecard or gain sufficient buy-in from business stakeholders to enable successful implementation. Meanwhile, previous investments in supplier relationship management (e.g., quarterly business reviews, annual executive summits, and the like) showed little return due to the lack of meaningful performance data.
The Solution
A structured scorecard, focused on the most critical measures of performance and value, combined with a comprehensive performance measurement and management process – one comprising clearly defined activities, and roles and responsibilities, for measuring both supplier and customer performance.
Diagnosing the Failure
• Because the scorecard included many dozens of metrics, the cost of gathering and analyzing supplier performance data was too high. In particular, busy clinical research staff who worked with suppliers claimed they did not have enough time to gather and report the data required to score suppliers.
• Not only did the scorecard include far too many metrics, they were not organized in natural or useful categories. Little thought had been given to how data would be used by individuals in contracting, project management, or senior management, and hence little attention had been paid to how best to structure and report supplier performance metrics.
• Consequently, it was difficult for anyone to make sense of the data that was (at least initially) collected.
• Moreover, in many cases, metrics were defined at too high a level of abstraction to be useful, e.g., “cycle time” or “contract adherence.”
• All of these flaws meant that senior management, who initially had requested supplier scorecards, did nothing with the results that began to be collected. Consequently, those who were being asked to shoulder the burden of increased measurement and reporting came to believe that it was a waste of their time to do so, and that since senior management didn’t care anyway, there would be no accountability for failing to comply. Testing this theory validated its correctness, at which point there was no incentive for busy people to take the time to gather data to enable scorecard reporting.
• Finally, despite useful thinking about supplier metrics, the project team spent very little time thinking about and defining the process (including specific roles and responsibilities for completing process steps and activities) associated with data gathering, analysis, reporting, and remediation or other actions suggested by performance data. This further undermined perceptions of the usefulness of data gathering and reporting (since no one had any idea how the data would be used), and left unaddressed issues of accountability for supplier performance measurement or management.
Background
The Strategic Sourcing Group at “MegaPharma,” a Fortune 500 pharmaceutical company, spent over a year trying to develop a scorecard to accurately assess their suppliers’ performance. Unfortunately, the initiative was a failure. Despite a great deal of good thinking and hard work, the cross-functional team who led the effort ended up producing an unwieldy scorecard that, within a few months of completion, was no longer used by anyone.
Approach
Vantage Partners partnered with MegaPharma to try again to develop a comprehensive, and consistent approach to supplier performance measurement and management. Working with a cross-functional team with representation from the Supplier Management Group in Strategic Sourcing, from clinical teams in all the therapeutic areas, from Finance, and from the Contracts group, Vantage first began by helping MegaPharma clarify the objectives for the overall performance management system in which the scorecard would be situated.
Overview of supplier performance management system objectives
• Gather and report data to enable effective selection of preferred suppliers
• Gather and report data to enable effective, and efficient, decision-making about project awards
• Solicit feedback from suppliers on how MegaPharma could improve its own operations, be a more desirable customer and business partner, and enable its suppliers to be more effective
• Identify and diagnose performance problems, and do so as early as possible, in order to enable effective remediation
• Build stronger, more committed relationships with suppliers
By clarifying and explicitly articulating performance management objectives, MegaPharma was in a better position to prioritize across hundreds of potential metrics and focus on what was most important to measure. Moreover, clarifying when and how performance metrics would be used helped the team determine the frequency of measurement and reporting (e.g., monthly, quarterly, end of project, and so on) for various metrics. Through a series of workshops, Vantage helped the MegaPharma team to hone a list of over 200 different metrics to a subset of the most critical and actionable. Each metric was evaluated based on the time and effort it would take to gather and report data, compared to the expected benefit to supplier performance and relationships, and ultimately to driving improved clinical research performance.
A holistic, and structured approach to scorecard design
The MegaPharma-Vantage team developed a scorecard designed to provide a holistic view of supplier performance on individual projects, of the health of its overall relationships with suppliers, and of the total value delivered by those relationships. The scorecard was organized around four key dimensions: operational performance, financial value, strategic value, and relationship quality (see a redacted summary of the scorecard in Figure 1 below).
Each dimension of the scorecard comprised several categories of metrics, such as innovation (as an element of strategic value) or level of trust (as an element of relationship quality). These categories were further broken down into three to five sub-categories, each of which might in turn be composed of anywhere from approximately one to five discrete metrics. A common scorecard framework, along with broad and consistent performance categories, allowed simple, high-level reports to be created for senior management across different kinds of suppliers, while at the same time ensuring that individual metrics (which often varied for different kinds of suppliers) were defined at a meaningful level of specificity. (See Figure 2)
In addition, each dimension of the scorecard included metrics against which suppliers were invited (and expected) to provide feedback to MegaPharma. Such two-way feedback was aimed at soliciting improvement opportunities from suppliers, at enabling more effective diagnosis of complex performance problems that were the result of how MegaPharma and its suppliers worked together (rather than a simple failure on the supplier’s part), and at building stronger relationships characterized by both accountability, and collaborative joint problem-solving.
Developing, and building buy-in to, a process for performance measurement and management
Recognizing that even the most carefully constructed metrics are useless without clearly defining how data will be gathered, analyzed, reported, and used, Vantage and MegaPharma developed a detailed process for performance measurement and management (a summary of the process is described below). To ensure that the process could actually be implemented, and that the wide range of stakeholders upon whom successful implementation would rely would support it, Vantage and the MegaPharma used a procedure called “blueprinting.” Over a series of about three months, the team iteratively developed and refined a series of drafts articulating, in concrete detail, specific activities and roles and responsibilities. Each iteration was reviewed with a broad cross-section of stakeholders who were asked for their concerns and suggestions, which were then incorporated into the next iteration of the blueprint draft.
Overview of supplier performance measurement and management process
Performance measurement and management activities occur at both the individual project level, and at the overall relationship level. The five activity streams that make up the supplier performance management process are:
1. Customize Scorecard — This activity occurs annually with each preferred supplier. The MegaPharma Supplier Relationship Manager (SRM) and the supplier’s relationship manager for MegaPharma review the standard supplier scorecard template, and jointly agree upon the specific metrics that will be used in light of the supplier’s specific capabilities, the projects the supplier will be working on, and the nature of the business arrangement with the supplier. (Note that some metrics are required for certain categories of suppliers, and some metrics are required across all suppliers.)
2. Gather Data — In this activity stream, three types of data are gathered in the following ways:
a. Objective, quantitative performance data about the supplier’s (and MegaPharma’s) performance on specific projects is captured from MegaPharma and supplier systems.
b. Subjective, project-level performance data (both quantitative and qualitative) is collected at regular intervals through surveys filled out by key stakeholders at MegaPharma and its suppliers.
c. Subjective data (both quantitative and qualitative) about overall supplier performance (not tied directly to specific projects), as well as data about relationship quality, is collected through periodic surveys filled out by key stakeholders at MegaPharma and its suppliers.
The MegaPharma and supplier relationship managers are responsible for managing the data collection process for both project-level and relationship-level performance information.
3. Analyze Performance — This activity occurs at two levels:
a. Project-level performance reports, focused on operational metrics, are created following the completion of each project (or every six months for those projects lasting longer than a year). Relationship managers and key team members from each project meet to develop project-specific remediation plans (if appropriate), and identify and document systemic improvement opportunities.
b. Reports on overall relationship quality (how well MegaPharma and its suppliers work together), and total value delivered by each supplier relationship, are created and jointly analyzed on a semi-annual basis by relationship managers and key executives on both sides.
4. Jointly Review Performance and Conduct Joint Planning — MegaPharma and supplier relationship managers meet quarterly to review cross-project operational performance. Semi-annually, they hold a strategic review and joint planning session to review overall relationship performance, develop corrective action plan(s) as needed, recognize and reward outstanding performance, capture and disseminate best practices, and identify and evaluate potential new value-generating opportunities for both MegaPharma and the supplier. Other stakeholders, in addition to the relationship managers, are involved in these meetings as appropriate (e.g. clinical research scientists, project managers, representatives from other functions at MegaPharma, etc.). After completion, the relationship managers communicate outcomes to the appropriate stakeholders in their respective organizations, oversee the execution of corrective action plans, and track progress on new opportunities.
5. Hold Supplier Portfolio Review Meeting — On an annual basis, a committee composed of senior executives at MegaPharma meets to review performance data and conduct comparative analysis of all preferred suppliers. Based on this analysis, the committee makes decisions to retain or replace specific suppliers on the preferred list, and develops long-range plans for the strategic use of suppliers, focused on new services or technologies they might provide, and also new business models that might be employed to structure arrangements with them. Finally, the committee evaluates internal improvement opportunities highlighted by the supplier performance measurement system.
The chart in Figure 4 provides a visual mapping of project-level and relationship-level activities between MegaPharma and its suppliers (detailed above) according to an annual calendar.
Lessons Learned
• Don’t try to measure too much. Measurement is a costly activity, trying to measure too much is a recipe for non-compliance.
• Consider imperfect, subjective or qualitative metrics (e.g., using surveys) that (often) provide 80% of the insight, with only 20 % of the data collection cost, versus more objective metrics (e.g., extracting and aggregating data from various electronic systems).
• After iterative development and refinement of scorecard metrics, sanity test the entire scorecard. Will the benefits outweigh the costs? What needs to be further pruned to make the system practical?
• Define exactly how data will be used. If there is not a compelling reason to have a metric to enable solving complex problems or making important decisions, don’t bother measuring it. This requires constructively challenging senior management or others who would “like to see a report that shows…” Such inclinations must be subjected to respectful scrutiny. “What decisions would such information enable you to make more effectively? Why? What actions would you take or not take based on different reported values for such metrics?”
• Define metrics with maximum specificity, including the unit of measure, where data will come from, and any calculations that need to be performed on raw data to generate the metric. “Cycle time” is not a meaningful metric. “Number of days from contract signed to first patient recruited” is.
• Don’t focus only on defining metrics. Spend as much or more time defining the process, not only for how data will be gathered and analyzed, but also for how data will be used, internally and with suppliers, to identify, diagnose, and address performance problems and improvement opportunities.
• Involve key suppliers in defining metrics, as well as designing the performance measurement and management system. They are an excellent source of insight. Moreover, their commitment is critical to success, and they are much more likely to be committed if you consult with them on development, than if you simply subject them to what you’ve decided on.
• Define and utilize metrics that focus on the value your key suppliers are getting out of the relationship. If you can’t easily replace a supplier, and even more so, if you expect them to innovate in various ways (which requires some level of investment on their part) then you need to pay attention not only to the value you are realizing, but what your suppliers are getting of the relationship as well. Otherwise the value you realize will be short-lived.
• Invest in behavioral skills training for those who will be involved in supplier performance management. At the end of the day, as important as data measurement is, continually improving performance depends critically on the ability of individuals to communicate effectively in the face of disagreement, and to diagnose and solve complex problems together.
About Vantage Partners, LLCA spin-off of the Harvard Negotiation Project,Vantage Partners helps companies achieve breakthrough business results by transforming the way theynegotiate with, and manage relationships with,their suppliers, customers, and alliance partners.