Proof of Concept (POC) in Spend Analytics

Roni Koskinen Aug 25, 2021

This blog is a part of our “Complete guide to RFP” blog series, where we share our insights regarding the RFP process. We at Sievo have participated in hundreds of RFPs in recent years and we wanted to ease your burden by sharing our knowledge on how to avoid misunderstandings and save time through a well-constructed and comprehensive RFP-process. See previous editions of our “complete guide to RFP” here:

Try before you buy – the golden rule for IT procurement

Every now and then, you may realize the following: you need to get out to the market since your solution status quo just doesn’t work, or your organization has outgrown the functionalities provided by your existing solution. It’s not a “walk in the park” as procurement systems in most cases require comprehensive solution configuration and change management in close collaboration with various stakeholders. This stakeholder input can have a drastic impact on system implementation success and user adoption. Implementation and adoption are the crucial success factors no one wants to fail at as they are likely to lead in uncaptured value of a newly introduced solution. Likewise, solutions solving complex problems with modern tools tend to be challenging to evaluate on paper. Fancy sounding AI algorithm names might seem promising and fulfil a column of your RFP questionnaire, but what really matters is the outcome – and those sophisticated methods (on paper) will not guarantee success.

So, how do you select the right Spend Analytics solution that fits your needs before making heavy resource commitments? How do you validate and communicate your decision making? Basing your decision solely on RFP questionnaire template doesn’t seem feasible, but there is a simple workaround. It follows the same logic as buying new running shoes: “Try before you buy”.

Proof-of-Concept: A limited scope implementation of a proposed solution with main purpose to verify feasibility of selected functional and non-functional aspects

A proof-of-Concept (or POC) is a great low-cost exercise when in need of further assurance in solution selection. POCs offer a limited scope implementation project with a limited timeline. It minimizes the risk in solution provider selection by allowing you to assess the solution fit according to your specific challenges, rather than relying on the provider’s sales promises. These projects also help you in building business cases as advanced analytics solutions potentially reveal tangible savings opportunities already within the POC scope. At minimum, you will gain a better understanding on resource requirements for full solution implementation based on realized work effort in the POC phase. This will help you justify your initial calculations. A POC enables evaluation of supplier’s ways of working and whether they would be a good fit with your teams. Further, you can use a POC to initiate a change management process for the soon-to-be-introduced new solution by involving a diverse set of stakeholders in the solution evaluation. POC is stakeholder expectations management at its best. These people could (and should) become your change ambassadors for the implementation project and improve the chances of successful project delivery and solution adoption down the line.

Your Proof-of-Concept is likely to fail

While there are undisputable benefits in trying before buying, sourcing teams fail all too often to achieve the goals they envisioned when entering a Proof-of-Concept with a potential procurement solution provider (or providers). Why? Many of these sourcing professionals forget there’s a significant difference between plug-and-play SaaS solutions, such as business communication platforms or ticketing systems, and solutions working with all but trivial challenges, like data harmonization in heterogenous system landscapes. While the plug-and-play SaaS software rarely calls for more than a free trial or demo access, the latter type of solution requires a well-planned evaluation process – and often a Proof-of-Concept project as part of it.

To put it simply, you are likely to fail in getting the desired outcomes of your Spend Analytics POC if it’s nothing more than a low-quality Excel data dump. Doomed-to-fail POCs usually look something like this. You invite several (four or more) solution providers taking part in your POC, just in case. You give them a lump of data that includes spend from all your 10 data sources over past 3 years. The providers are given a one-week deadline for submission, with no further guidance. There are only 30 minutes reserved to showcase the POC results at the end (in an online meeting, naturally). During the presentation half of the team (who have no experience of Spend Analytics other than in Excel pivots) zone out while familiarizing themselves with the POC background material that they didn’t have time to absorb before the meeting, and there’s no time left to discuss the actual value add and capabilities of the provider.

To generalize the issues above, there are several common mistakes made by sourcing professionals:

  • Not defining important provider capabilities or sharing, but only consulting a few internal stakeholder groups and sharing no expectations or evaluation criteria with the provider.
  • Allow only showcasing a limited scope of activities while excluding challenging tasks, such as extraction of ERP data, that considerably affect end-result of subsequent activities
  • Expecting all functionalities to be available and fully configured to your needs – even those you’re only slightly interested in
  • Providing too short of a timeline for execution from data delivery to results presentation while still expecting similar results as from full project

  • Including an unreasonable amount of data considering the envisioned timeline (there should still be enough data to evaluate provider’s capabilities to manage the system landscape)
  • Involving too many providers in the Proof-of-Concept phase and thereby (or otherwise) allocating too little time to support each supplier in solution configuration related questions
  • Involving too few internal stakeholders with homogeneous backgrounds, and in the worst case, no-one with hands-on experience in solving the challenges the solution is envisioned to solve
  • Forgetting to properly evaluate the results of a POC – it requires more than an online presentation to assess the solution and scoring of functionalities.

Do any of these mistakes sound familiar? Yep, and you’re not alone. In fact, 90% of procurement technology implementations fail (CIPS) and POC projects are no different. Now, if you decide where to put your efforts and avoid belonging to that majority, POC projects just make sense. Not only because they have smaller scope and are easier to manage, but their results will also directly influence the likelihood of (potential) full implementation project success.

Continue reading to learn how to overcome the common pitfalls in POC following only three simple steps!

The 3 steps towards successful POC projects

While there are numerous common mistakes one may make, there are only 3 steps you need to take to get your POC right. These steps will essentially ensure the POC answers the questions you wanted to be answered while also maintaining the low-cost and short delivery timeline of the project. Make sure to pay attention to the next pieces of advice to master the art of trying before buying.

First, you need to define of what needs to be proven, or in other words, (1) define criteria for success. Since everyone can agree that there are not infinite resources available for solution evaluation nor time allocated for Proof-of-Concept delivery prior to evaluation, it is crucial that the resources and time available are used efficiently and actions are focused on essential areas. These might be features or services previous solutions (internal or external) have failed to deliver upon, or new more advanced functionalities where solution value or provider maturity must be evaluated carefully. Then, one should source for evaluation criteria from different stakeholder groups (category managers, buyers, procurement analysts, procurement controlling, IT, etc.) and from people preferably with hands-on experience to avoid missing crucial areas to consider in evaluation process. This will also initiate a broad change management process across the company instead of limiting it to a narrow sourcing team. Once you’ve ironed the criteria out, remember to share those with your candidates so that they know where to put their efforts.

Next, you need to (2) align the POC scope with the defined success criteria to ensure all important features and functionalities are demonstrated adequately and the focus is crystal clear to maintain a limited scope and timeline. The scope can be split to activities, data, and functionalities. Let’s cover these in more detail as they will have a significant impact on how the project will look (and if you will be able to get evidence for your evaluation).

  • Major activities in Spend Analytics include connectivity (extraction and integration of data from multiple sources), cleansing and enrichment (classification, supplier parenting, and connecting internal and external data sets, for example), and analytics. Often, only some of these are included in POC scope. The most common combinations of these are only analytics, classification, and analytics, and only classification. Have you faced challenges in extracting Procurement relevant data from your ERP systems? If you recognize the struggle, would it make sense to test the provider’s capabilities in that area instead of providing them with the same poor data set for analysis you already have at hand even if it could be improved? It should be a no-brainer, since data extract quality influences the outcome of many subsequent activities like classification, supplier parenting, or analytics and insights.
  • In spend extraction and classification more is more, or more data means more integrations and more to classify, and thereby longer lead time for a project. While AI streamlines tail spend classification and the more data is in scope the more economies and scale will be leveraged, an additional source system – with its own master data structures and value – in the POC scope will increase the workload to train the AI engine by human classifications. Further, it is hard if not impossible to focus your own validation efforts systematically to different providers’ end-results in the evaluation phase with the limited internal resources available. The providers might also have put their own focus to different categories to showcase their best performance while coping with a limited timeline which makes their end-results non-comparable in the end.

So, is your data dirty only in a single business entity or only within indirect categories? Perhaps you would like to test vendor capability in data cleansing and enriching in that area only. Working with a limited scope can both shorten the timeline and make providers more comparable while still allowing them to involve all relevant internal stakeholders in the process.

  • Advanced best-of-breed Spend Analytics often offer a broad set of analytics functionalities combined with additional modules and features, such as savings tracking or benchmarking, for end-to-end value delivery in strategic procurement. Implementing all such modules and features in a POC might be an overkill especially when they require extensive customer-specific configuration work. Like in the data scope, it is a good place to challenge the relevance of non-core functionalities in POC scope. You might ask: “What are the solution functionalities that require being thoroughly tested with your own data, processes and users?” or “What am I buying and what am I only interested in?” You could consider if the provider’s demo solution is enough for testing certain modules or functionalities. This would allow you to focus on the most important areas in the project instead of burning calories in non-value adding activities.

Finally, and most importantly, (3) you should set up a predefined and well-documented evaluation process that is fit-to-purpose. This sounds like a trivial exercise, but it isn’t (at least after having seen dozens of evaluations only or mostly based on a quick presentation). To begin with, you need to get back to your success criteria which should outline the key themes of the evaluation together with more detailed and refined evaluation points. What should one then evaluate? Well, you should put emphasis on the team’s first impressions and perceived user experience. These will influence user adoption in the wider user-base in full rollout later down the road. Furthermore, it is also worthwhile to assess non-functional aspects such as if the tool aligns with your organizational strategy, processes, and culture, or if the provider is willing to develop the functionalities and features together with you.

Nevertheless, don’t forget the key challenges you wanted to test in the first place with the POC, e.g. enhance quality and integrity of data extracted from your ERPs, improve classification accuracy and coverage, or finding those automated, truly actionable insights from analytics. While top of the pack UX is vital or willingness, a new and shiny solution will take you nowhere from the status quo if it replicates the original issues like low data quality.

Evaluation itself is easily arranged by 1) booking enough time from the stakeholders participating and 2) setting up an Excel, a simple and powerful tool to facilitate collecting feedback. It is essential to involve stakeholders from same diverse backgrounds as you did for evaluation criteria collection. As a project manager you should make sure your evaluators will remain engaged in the process and that they will take part in required briefings and presentations. Weighted average scoring is a powerful tool to ensure the total score will suitably drive decision making in the end. However, since grades-only evaluation gives a relatively “black and white” picture, they should be accompanied with rich qualitative feedback to also consider the subtle differences in providers’ offerings and ways of working. With these two forms of feedback combined, you avoid the tick-the-box feature bingo that will lead to failure in vendor selection.

Now, long story short: “Well planned is half done” applies very well to POC projects and the outlined three simple steps will get you there.

Spot-on assurance for accelerated time-to-value

Following the steps, you should have quick results that support your decision-making, internal sales, and change management. You should have focused on areas that were important in the first place. You should also have a better understanding of solution provider’s ways of working and whether they would be a good fit for your organization. In short, you should be able to make a well-informed decision in a timely manner.

Are you looking for a Spend Analytics solution and planning for a Proof-of-Concept process? Let us know and let’s have a chat!

Contact Sales

Banner image by mikar (Unsplash.com)

SA101 Mockup

Begin your Procurement Analytics journey now!

Get your procurement analytics journey going with our free Spend Analysis 101 eBook. No matter if you're a Procurement Analytics Pro or a beginner in the field, this book will help you develop and improve the procurement analytics within your procurement organization.

Never miss news from Sievo

Get updates straight to your inbox