Measuring the benefits of your service
Building the wrong thing can be very costly. It wastes users바카라 사이트� time, causes frustration and leads to people getting in touch via less cost effective channels.
It바카라 사이트檚 sometimes hard to make the case for building the right thing - especially if you바카라 사이트檝e been funded to address a problem in a specific way.
The best way to overcome this is to explore different options and make sure you바카라 사이트檝e got a deep understanding of:
- the benefits a particular solution delivers
- how those benefits help address wider departmental or governmental objectives
You can use the fact that you바카라 사이트檙e working in an agile way - and are constantly learning about your users and their behaviour - to your advantage.
Benefits realisation is a team sport, but it바카라 사이트檚 usually owned by a product manager or service owner.
If you바카라 사이트檙e looking to build a business case for spend approval from HM Treasury (HMT), refer to the Green Book guidance.
Discovery: spot problems and estimate how much they바카라 사이트檙e costing
Your discovery research should help you understand who your users are and identify problems in the area you바카라 사이트檙e working in. These could be things like:
- poor user experiences that cause people to use less cost effective channels
- inefficient or time consuming processes
- legacy systems that need replacing
As you do this, it바카라 사이트檚 useful to work out how big the problems are - how long it takes users to do a thing or how much a process is costing, for example.
It바카라 사이트檚 important to establish this baseline figure. You can only know if you바카라 사이트檝e improved things by the end of your project if you know where you바카라 사이트檙e starting from.
To achieve this, your user researcher should work with an analyst (an economist or business analyst, for example) to come up with research questions that will help you quantify these things.
Your team can combine your research findings with insights from desk-based research (like academic articles or ONS data) to build a clearer picture of the size of the problem.
You can to weigh up costs against benefits to users and to help you understand:
- the problems you바카라 사이트檙e solving
- if you바카라 사이트檙e providing value to users
It바카라 사이트檚 fine if you바카라 사이트檙e using assumptions to make estimates at this point. You바카라 사이트檒l have the chance to test these assumptions as you move through the development phases.
Download an example of a team바카라 사이트檚 .
Estimate how much it바카라 사이트檒l cost to run an alpha
Once you바카라 사이트檝e got an idea of the biggest problems, you can start thinking about how much it might cost to put a team together and run an alpha.
When you바카라 사이트檙e estimating, think about the number of people you might need and their salary costs, whether you바카라 사이트檒l need any non-civil servant support and any software, hosting or technology costs you바카라 사이트檙e likely to incur.
Download an example of for alpha.
Alpha: test different solutions
Having spotted and sized problems you think are worth solving, you should spend the alpha phase exploring how to solve them.
Test different ideas and scrutinise your riskiest assumptions. This should help you figure out how difficult it바카라 사이트檇 be to deliver those ideas for real and understand the benefits each solution would deliver.
At this stage, it바카라 사이트檚 still okay to base estimations of benefits on assumptions. You바카라 사이트檒l gather data throughout your alpha that바카라 사이트檒l help you test and validate them.
Any of the following sorts of things would be considered benefits.
Cashable benefits
Cashable benefits are changes that result in your organisation having more money to spend, either through savings or through additional revenue.
Non-cashable benefits
Non-cashable benefits are changes that don바카라 사이트檛 lead to an immediate benefit, but address problems you바카라 사이트檇 have had to fix eventually. They include things like saving money in future budgeting periods, or avoiding future procurement costs.
Wider economic benefits
These improve things for your users or other groups outside your organisation and include things like:
- saving users time or improving their experience
- reducing private sector costs or the burden on charities
Benefits don바카라 사이트檛 have to be financial. It바카라 사이트檚 okay to focus on realising qualitative benefits - especially if your end goal is something like improving user experience.
Show how fixing the problems would address wider strategies
During your alpha, it바카라 사이트檚 useful to think about what might happen if you were able to fix one or more of the problems you identified during discovery. And, if possible, how addressing them might help your organisation meet its wider strategic objectives.
This helps you understand the wider context you바카라 사이트檙e working in and demonstrate the value of your work to stakeholders.
One way of doing this is to create a benefits map. This helps you track the impact of fixing some of the problems and the changes this might bring about.
Creating a benefits map
Follow these steps to put your map together.
-
Get your team and stakeholders together.
-
Identify the wider departmental goal or strategy your project has been set up to address. This is known as your 바카라 사이트榮trategic objective바카라 사이트�.
-
Work out the things you바카라 사이트檇 need to do to meet that strategic objective - for example making a process more secure or easier to use. These things are known as your 바카라 사이트榚nd benefits바카라 사이트�.
-
Figure out what you바카라 사이트檇 need to do to realise your end benefits - for example improving accessibility. These things are known as 바카라 사이트榠ntermediate benefits바카라 사이트�.
-
Identify the changes you바카라 사이트檇 need to bring about to make your intermediate benefits possible. For instance, to improve accessibility, you might need to develop a new user interface. These are known as 바카라 사이트榚nabling changes바카라 사이트�.
-
List some immediate, tangible deliverables for your project. For example, building a new system.
-
List the problems or opportunities you identified at discovery. These are your 바카라 사이트榙rivers바카라 사이트� and could include things like low user satisfaction.
You can then plot these benefits into a flow diagram, starting with your drivers and ending with your strategic objective, to show the knock-on effect of your work.
It바카라 사이트檒l be much easier to make the case for doing your work a certain way if your solution not only makes things better for users, but helps government deliver the things it바카라 사이트檚 committed to deliver.
Create an economic model at the end of alpha
By the end of alpha, you바카라 사이트檒l have an understanding of the size of the problem and a few potential solutions.
Create an economic model to help you:
- summarise what you바카라 사이트檝e learned up to this point
- choose which solution you take forward to beta
The economic model should show an estimate of the costs and benefits of your chosen solution in beta and in live.
There are a few principles you should follow when doing this.
Show how much you바카라 사이트檒l improve things by
Work out the difference between the baseline figure you identified during discovery and your estimate of how much you might be able to improve things by.
Repeat this process for each of the solutions you explored at alpha.
For example, if it바카라 사이트檚 currently taking users an average of 45 minutes to submit a claim, but your work will lead to it taking 15 minutes, then you바카라 사이트檙e saving users 30 minutes on average.
Or, if a process is currently costing 拢10 per transaction and you think it might cost 拢4 per transaction once you바카라 사이트檝e finished your project, then the benefit is 拢6 per transaction.
It바카라 사이트檚 human nature to overestimate things. This is called 바카라 사이트榦ptimism bias바카라 사이트�.
Make sure to be realistic about what benefits you can deliver given your budget and the amount of time you바카라 사이트檝e got. This might mean saying you바카라 사이트檒l deliver a smaller number of features by public beta than you think you could if things went perfectly.
It바카라 사이트檚 also important to think about what happens if things don바카라 사이트檛 go to plan once you바카라 사이트檙e in public beta. So make sure to think about all possible outcomes. For example, what would happen if:
- take up of your service was as low as it could conceivably be
- take up of your service was as high as it could conceivably be
This is known as 바카라 사이트榮ensitivity analysis바카라 사이트�.
Scrutinise your assumptions
The second part of sensitivity analysis involves scrutinising the assumptions your predictions are based on. This is particularly important early on in a project when your assumptions might not be based on much research.
For instance, predicting that you바카라 사이트檒l save 拢6 per transaction could be based on assuming that:
- it바카라 사이트檒l take a certain amount of time for contact centre staff to deal with an application or claim
- a certain number of users will need to contact you via offline channels
Tweak some of these numbers and see how this affects your economic model. For instance, what would your cost per transaction be if contact centre staff were able to process claims in 5 minutes rather than 10?
Focus on scrutinising your riskiest assumptions. Include this sensitivity analysis in your economic model.
Estimate how much your team will cost in beta and live
Include your estimated delivery costs for each of your potential solutions. This means thinking about the team you바카라 사이트檒l need and any other costs (such as hosting or licensing) you바카라 사이트檇 incur.
Sometimes projects are impacted by issues you couldn바카라 사이트檛 possibly have predicted. So it바카라 사이트檚 worth overestimating how much a project will cost or how long it바카라 사이트檒l take to deliver.
Download an for beta and live.
Advocate for the right solution
Following this process can help you make the case for building the right solution, especially if you바카라 사이트檝e been given a project brief you don바카라 사이트檛 agree with.
It gives you a firm idea of the costs and benefits of your potential solution, which you can compare to the thing you바카라 사이트檝e been asked to do.
Beta and live: monitor and evaluate your project
Agree a set of metrics to track at the start of beta (including benefits) with stakeholders. These could be things like the number of people using your service, volumes or cost and should relate to the end benefits and strategic objectives you listed in your benefits maps.
In order to monitor and evaluate benefits being delivered you may want to gather data from your users through:
- user research
- case studies
- questionnaires
- analytics data
Make sure to ask the questions in a way that means the user only gives you information about the benefits of your project.
This will help you demonstrate how the benefits compare to what you forecast in your end of alpha economic model.
You should have decided how you바카라 사이트檙e going to gather data before you move into private beta - and should make data gathering and iteration a constant part of the beta and live stages.
Iterate your models
Working in an agile way means you can test things and get rich data fairly quickly - especially in beta and live when your service is being used by a high number of users.
Feed your findings back into your economic model and benefits map after each iteration period. And turn any success stories into case studies - these are an excellent way of showing the value of your service, especially to stakeholders.
Decide whether you should continue your project
At the end of beta you should have an iterated economic model and benefits map that demonstrate the value for money your service will deliver if progressed into live.
You should also have data showing how the benefits compare with the forecasts outlined in your end of alpha economic model. Support this data with case studies and user research findings.
All of this information should help establish whether it바카라 사이트檚 worth continuing your project into live, or whether you should stop or change what you바카라 사이트檙e doing.
- Last update:
-
Guidance first published