Make your next car electric: a behavioural science messaging trial
Published 1 February 2021
© Crown copyright 2021
This publication is licensed under the terms of the Open 바카라 사이트 Licence v3.0 except where otherwise stated. To view this licence, visit or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at /government/publications/behavioural-science-messaging-trial-make-your-next-car-electric/make-your-next-car-electric-a-behavioural-science-messaging-trial
Introduction
The Department for Transport (DfT) worked with the Driver and Vehicle Licensing Agency and the Cabinet Office to run a behavioural trial to investigate messaging to best encourage electric vehicle uptake, supporting the government바카라 사이트™s net-zero ambitions. The trial commenced on 6 March 2020 and ran for 130 days, taking place during the government바카라 사이트™s consultation on bringing forward the phase-out date for the sale of new petrol, diesel and hybrid cars and vans (February to July 2020).
The trial targeted people who had just renewed their car tax online by implementing a message that encouraged users to click through to the Go Ultra Low (GUL) website, which encourages electric vehicle purchase.
A total of 8 messages (7 test messages and a control) were compared. Messages were developed by DfT바카라 사이트™s in-house behavioural scientists, and policy and communications colleagues from the Office for Low Emission Vehicles (OLEV). Content ranged from cost savings to environmental and air pollution benefits. Messages were compared to see which produced the highest click-through rate to GUL.
The most successful message related to the government바카라 사이트™s consultation on ending the sale of internal combustion engine vehicles:
The 바카라 사이트 are consulting on ending the sale of new petrol, diesel and hybrid cars and vans by 2035 or earlier. Are you ready? Make your next car electric.
The next most effective message was one that appealed to social norms:
Join the 6,000 new drivers every month who make the switch to an electric vehicle. Make your next car electric.
The message around the number of deaths related to air pollution also performed well, but to a lesser extent than the previous 2:
Between 28,000 and 36,000 people die every year as a result of air pollution. What you drive makes a difference. Make your next car electric.
All messages, including the control, contributed to increased traffic to GUL. Scaling the findings from this study up, it is estimated that:
- the control message (바카라 사이트˜Information is available on electric vehicles. Make your next car electric.바카라 사이트™) would produce about 350,300 annual click-throughs to GUL
- that 바카라 사이트˜The 바카라 사이트 are consulting on ending the sale of new petrol, diesel and hybrid cars and vans by 2035 or earlier. Are you ready? Make your next car electric.바카라 사이트™ would result in around 419,900 annual click-throughs to GUL
- that 바카라 사이트˜Join the 6,000 new drivers every month who make the switch to an electric vehicle. Make your next car electric.바카라 사이트™ would generate around 407,900 more annual click-throughs to GUL
Three messages performed worse than the control and were therefore unsuccessful. The lowest click-through rate was produced by the message around rapid charge point availability:
Rapid charge points for electric vehicles are available at almost all motorway service stations in the UK. Make your next car electric.
The second-lowest click-through rate was produced by the message around the ease of charging:
Charging your electric vehicle at home can be as easy as charging your phone overnight. Make your next car electric
The third unsuccessful message (although to a lesser extent) was around the cost per mile of using an electric vehicle:
Fully electric vehicles could cost from as little as 1p per mile to run 바카라 사이트“ less than a quarter of the cost of the most fuel-efficient petrol or diesel vehicles. Make your next car electric.
The message around traffic and carbon emissions performed similarly to the control: 바카라 사이트˜Road traffic is the biggest single contributor to carbon emissions in the UK. What you drive makes a difference. Make your next car electric.바카라 사이트™
COVID-19 impact
Click-through rates of each message was compared from pre-lockdown to post-lockdown. Overall, patterns in the trial are similar across both periods. However, during lockdown, click-through rates for all messages decreased significantly by an average of 14%.
This suggests that people were less receptive to all messages during this period. 바카라 사이트˜Are you ready?바카라 사이트™ continued to have the best click-through rate overall. 바카라 사이트˜Are you ready?바카라 사이트™ and 바카라 사이트˜6,000 new drivers바카라 사이트™ performed better than the control pre- and post-lockdown. 바카라 사이트˜28,000 to 36,000 deaths바카라 사이트™ only performed better than the control post-lockdown. The success of messages varied through lockdown, highlighting the importance of message context. It is therefore important to continue developing and testing messages to ensure we apply messages correctly.
Messages
Message background
The messages that were tested were developed by DfT바카라 사이트™s in-house behavioural scientists, and policy and communications colleagues from the OLEV. The content of the messages ranged from cost savings to environmental and air pollution concerns.
The messages were hosted on the 바카라 사이트 Tax your vehicle service. After successfully renewing car tax online, users were presented with a 바카라 사이트˜Thank you바카라 사이트™ page. A single message was displayed randomly to each user on this page and all linked to GUL.
Message content, summary click-through rates and shorthand
Table 1: message content, click-through rates and shorthand descriptor
|
Message |
Content |
Click through rate |
Shorthand |
|---|---|---|---|
| Test E |
The 바카라 사이트 are consulting on ending the sale of new petrol, diesel and hybrid cars and vans by 2035 or earlier. Are you ready? |
3.50% |
바카라 사이트˜Are you ready?바카라 사이트™ |
| Test B |
Join the 6,000 new drivers every month who make the switch to an electric vehicle. . |
3.40% |
바카라 사이트˜6,000 new drivers바카라 사이트™ |
| Test D |
Between 28,000 and 36,000 people die every year as a result of air pollution. What you drive makes a difference. . |
3.10% |
바카라 사이트˜28,000 to 36,000 deaths바카라 사이트™ |
| Test C |
Road traffic is the biggest single contributor to carbon emissions in the UK. What you drive makes a difference. . |
2.93% |
바카라 사이트˜Traffic and carbon emissions바카라 사이트™ |
|
Control |
Information is available on electric vehicles. . |
2.92% |
Control |
| Test A |
Fully electric vehicles could cost from as little as 1p per mile to run 바카라 사이트“ less than a quarter of the cost of the most fuel-efficient petrol or diesel vehicles. . |
2.80% |
바카라 사이트˜1p per mile바카라 사이트™ |
| Test F |
Charging your electric vehicle at home can be as easy as charging your phone overnight. . |
2.74% |
바카라 사이트˜Ease of charge바카라 사이트™ |
| Test G |
Rapid charge points for electric vehicles are available at almost all motorway service stations in the UK. . |
2.36% |
바카라 사이트˜Charge point availability바카라 사이트™ |
Results and findings
Which message was most effective?
Click-through rates were compared to identify which message produced the highest click-through rate overall and which had performed significantly better than the control and other messages.
Comparisons were made using data over 130 days (6 March to 14 July 2020). In total, there were 4,272,526 tracked users of the government website for paying vehicle tax. On average, there were 534,066 users per condition/message.
Table A in the annex presents a breakdown of how many users saw each test message, how many clicked through, and what the click-through rate was for each message. Chart 1 highlights the click-through rates of each message compared with the control.
Chart 1: percentage of visitors to the government vehicle tax payment website that clicked through to GUL for each message for the whole trial (130 days from 6 March to 14 July 2020) (%). 바카라 사이트˜Are you ready?바카라 사이트™ performed best.
| Message | Click-through rate (%) |
|---|---|
| Control | 2.92 |
| Charge point availability | 2.36*** |
| Ease of charge | 2.74*** |
| 1p per mile | 2.8** |
| Traffic and carbon emissions | 2.93 |
| 28,000 to 36,000 deaths | 3.1*** |
| 6,000 new drivers | 3.4*** |
| Are you ready? | 3.5*** |
In the chart and data table above, N=4,272,526. ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Note
For each chart and table in this guidance:
- where N is reported, N represents the number of participants
- Z-test is a test of the probability that a difference between two groups means (averages) is a chance (false positive) result
- a p-value indicates the probability out of 1 that a result is due to chance, a significance level indicates the maximum probability that a result is due to chance
Effectiveness of messages over the entire trial (130 days)
Table B in the annex shows a breakdown of comparisons made. 바카라 사이트˜Are you ready?바카라 사이트™ produced the highest click-through rate overall and was significantly higher than all other messages. 바카라 사이트˜6,000 new drivers바카라 사이트™ was the next most effective message, and 바카라 사이트˜28,000 to 36,000 deaths바카라 사이트™ produced the third-highest click-through rate. Both were significantly higher than the control message.
Three messages resulted in significantly fewer click-throughs than the control: 바카라 사이트˜Charge point availability바카라 사이트™ produced the lowest click-through rate, then 바카라 사이트˜Ease of charge바카라 사이트™ followed by 바카라 사이트˜1p per mile바카라 사이트™.
Assessing impact
Over the 130 days that this trial ran for, there were 4,272,526 users of the Tax your vehicle service. This means, an average of around 32,800 users a day. Using this value, we can anticipate around 12 million users a year.
By using the anticipated number of users of the Tax your vehicle service, the click-through rate for the different messages can be used to estimate the potential increase in click-throughs to GUL over a year.
The estimates are:
- control (2.92%), about 350,200 click-throughs
- 바카라 사이트˜Are you ready?바카라 사이트™ (3.50%), about 419,900 click-throughs
- 바카라 사이트˜6,000 new drivers바카라 사이트™ (3.40%), about 407,900 click-throughs
- 바카라 사이트˜28,000 to 36,000 deaths바카라 사이트™ (3.10%), about 371,900 click-throughs
Did COVID-19 lockdown measures impact results?
Lockdown measures put in place to protect the public from COVID-19 may have impacted public attitudes and click-through behaviour.
Further analysis was conducted to explore whether this affected the message testing results by comparing which message generated the highest click-through rate pre-lockdown (6 to 23 March 2020; 17 days) and which generated the highest click-through rate post-lockdown (24 March to 14 July 2020; 113 days).
Chart 2: comparison of click-through rates pre- and post-lockdown for each message (130 days from 6 March to 14 July 2020). Generally, post-lockdown click-through rates were statistically lower than pre-lockdown.
| Message | Pre-lockdown | Post-lockdown |
|---|---|---|
| Are you ready? | 4.06 | 3.4*** |
| 6,000 new drivers | 3.64 | 3.35*** |
| 28,000 to 36,000 deaths | 3.41 | 3.02*** |
| Traffic and carbon emissions | 3.42 | 2.92*** |
| Control | 3.32 | 2.86*** |
| 1p per mile | 3.22 | 2.76*** |
| Ease of charge | 3.25 | 2.68*** |
| Charge point availability | 2.73 | 2.3*** |
In the chart and data table above, pre-lockdown N= 440,269; post-lockdown N=3,828,349. ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Overall, lockdown reduced the click-through rate of all messages by an average raw percentage point of 0.47%. This is an average proportional decrease of 14% and was significant for all messages (see chart 2). 바카라 사이트˜Are you ready?바카라 사이트™ decreased the most from pre- to post-lockdown (4.06% to 3.40%), and 바카라 사이트˜6,000 new drivers바카라 사이트™ the least (3.64% to 3.35%) (see table C in the Annex).
See tables D to G in the annex for comparisons and breakdown of how many users saw each test message, how many clicked through, and what the click-through rate for each message was pre- and post-lockdown. See chart 3 and chart 4 for how each message performed against the control.
Pre-lockdown, 바카라 사이트˜Are you ready?바카라 사이트™ performed significantly better than the control and the rest of the messages. 바카라 사이트˜6,000 new drivers바카라 사이트™ was also effective and performed significantly better than the control but to a lesser extent. No other messages were significantly better than the control.
Chart 3: percentage of visitors to the government vehicle tax payment website that clicked through to GUL for each message pre-lockdown (%) (17 days between 6 to 23 March 2020). 바카라 사이트˜Are you ready?바카라 사이트™ performed best.
| Click-through rate (%) | |
|---|---|
| Control | 3.32 |
| Charge point availability | 2.73*** |
| Ease of charge | 3.22 |
| 1p per mile | 3.25 |
| Traffic and carbon emissions | 3.41 |
| 28,000 to 36,000 deaths | 3.42 |
| 6,000 new drivers | 3.64** |
| Are you ready? | 4.06*** |
In the chart and data table above, N=440,269. ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Post-lockdown, the top-performing messages were 바카라 사이트˜Are you ready?바카라 사이트™, 바카라 사이트˜6,000 new drivers바카라 사이트™ and 바카라 사이트˜28,000 to 36,000 deaths바카라 사이트™. 바카라 사이트˜Are you ready?바카라 사이트™ and 바카라 사이트˜6,000 new drivers바카라 사이트™ both performed similarly to one another and were significantly better than the control. 바카라 사이트˜28,000 to 36,000 deaths바카라 사이트™ was significantly worse than the 2 top-performing messages, but still performed better than the control.
Chart 4: percentage of visitors to the government vehicle tax payment website that clicked through to GUL for each message post-lockdown (%) (113 days from 24 March to 14 July 2020). 바카라 사이트˜Are you ready?바카라 사이트™ and 바카라 사이트˜6,000 new drivers바카라 사이트™ performed best.
| Click-through rate (%) | |
|---|---|
| Control | 2.86 |
| Charge point availability | 2.3*** |
| Ease of charge | 2.68*** |
| 1p per mile | 2.76** |
| Traffic and carbon emissions | 2.92 |
| 28,000 to 36,000 deaths | 3.02*** |
| 6,000 new drivers | 3.35*** |
| Are you ready? | 3.4*** |
In the chart and data table above, N=3,828,349. ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Annex
Cookie compliance policy
In accordance with General Data Protection Regulations (GDPR), 바카라 사이트 has a cookie compliance policy. For this reason, we are unable to track all users to 바카라 사이트. This means numbers from this trial are likely to be lower than the total number of visits to the Tax your vehicle service.
Unique page views, click-throughs to GUL and click-through rates were tracked using Google Analytics. Due to the way data is processed and stored in Google Analytics, numbers are often rounded up or down. This causes minor inaccuracies in unique views and click-throughs when evaluating different data periods. This results in varying total page views in our analysis. Click-through rate is unaffected by this.
Tables
Table A: breakdown of unique views to the government tax payment website, click-throughs to GUL and click-through rate for each message for the entire trial (130 days from 6 March to 14 July 2020)
| Message | Unique views of the Tax your vehicle page | Click-throughs to GUL | Click-through rate (%) |
|---|---|---|---|
| A | 535,918 | 15,019 | 2.8 |
| B | 531,965 | 18,081 | 3.4 |
| C | 533,830 | 15,624 | 2.93 |
| D | 535,189 | 16,597 | 3.1 |
| E | 535,575 | 18,729 | 3.5 |
| F | 533,574 | 14,602 | 2.74 |
| G | 536,729 | 12,669 | 2.36 |
| Control | 529,746 | 15,493 | 2.92 |
Table B: percentage point difference in click-through rate between Control, Message E and other trial messages (130 days from 6 March to 14 July 2020)
| Message | Control | Message B | Message D |
|---|---|---|---|
| A | -0.12 *** | -0.60 *** | -0.30 *** |
| B | + 0.48 *** | n/a | +0.30 *** |
| C | +0.01 (p=0.950) | -0.47 *** | -0.17 *** |
| D | +0.18 *** | -0.30 *** | n/a |
| E | +0.58 *** | +0.10 ** | +0.40 *** |
| F | -0.18 *** | -0.66 *** | -0.36 ** |
| G | -0.56% *** | -1.04 *** | -0.74 *** |
| Control | n/a | -0.48 *** | -0.18 *** |
In the table above, ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Table C: comparisons of click-through rates pre-lockdown versus post-lockdown for each message (130 days from 6 March to 14 July 2020)
| Message | Pre-lockdown click-through rate (%) | Post-lockdown click-through rate (%) | Proportional decrease (%) |
|---|---|---|---|
| A | 3.22 | 2.76 | 14.29 |
| B | 3.64 | 3.35 | 7.97 |
| C | 3.42 | 2.92 | 14.62 |
| D | 3.41 | 3.02 | 11.44 |
| E | 4.06 | 3.4 | 16.26 |
| F | 3.25 | 2.68 | 17.53 |
| G | 2.73 | 2.3 | 15.75 |
| Control | 3.32 | 2.86 | 13.86 |
Table D: breakdown of unique views of the government tax payment website, click-throughs to GUL and click-through rate for each message pre-lockdown (17 days from 6 to 23 March 2020)
| Message | Unique views of the Tax your vehicle page | Click-throughs to GUL | Click-through rate (%) |
|---|---|---|---|
| A | 55,110 | 1,772 | 3.22 |
| B | 54,959 | 1,999 | 3.64 |
| C | 55,037 | 1,880 | 3.42 |
| D | 55,261 | 1,882 | 3.41 |
| E | 55,449 | 2,252 | 4.06 |
| F | 54,563 | 1,775 | 3.25 |
| G | 55,238 | 1,509 | 2.73 |
| Control | 54,652 | 1,817 | 3.32 |
Table E: percentage point difference in click-through rate between Control, Message E and other trial messages (17 days from 6 to 23 March 2020)
| Message | Control | Message E |
|---|---|---|
| A | - 0.10 (p=0.310) | -0.84 *** |
| B | + 0.32 ** | -0.42 *** |
| C | +0.10 (p=0.400) | -0.64 *** |
| D | +0.09 (p=0.460) | -0.65 *** |
| E | +0.74 *** | n/a |
| F | -0.07 (p=0.510) | -0.81 *** |
| G | -0.59 *** | -1.33 *** |
| Control | n/a | -0.74 *** |
In the table above, ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Table F: breakdown of unique views to the government tax payment website, click-throughs to GUL and click-through rate for each message post-lockdown (113 days from 24 March to 14 July 2020)
| Message | Unique views of the tax payment website | Click-throughs to GUL | Click-through rate (%) |
|---|---|---|---|
| A | 479,276 | 13,225 | 2.76 |
| B | 476,394 | 15,954 | 3.35 |
| C | 479,113 | 13,988 | 2.92 |
| D | 479,055 | 14,461 | 3.02 |
| E | 479,991 | 16,312 | 3.40 |
| F | 477,909 | 12,794 | 2.68 |
| G | 480,580 | 11,043 | 2.30 |
| Control | 476,031 | 13,630 | 2.86 |
Table G: percentage point difference in click-through rate between Control, Message B, Message E and other trial messages (113 days from 24 March to 14 July 2020)
| Message | Control | Message B | Message D | Message E |
|---|---|---|---|---|
| A | -0.10** | -0.59*** | -0.26*** | -0.64*** |
| B | + 0.49*** | n/a | +0.33*** | -0.05 (p=0.180) |
| C | +0.06 (p=0.100) | -0.43*** | -0.10** | -0.48*** |
| D | +0.16*** | -0.33*** | n/a | -0.38*** |
| E | +0.54*** | +0.05 (p=0.180) | +0.38*** | n/a |
| F | -0.20*** | -0.67*** | -0.34*** | -0.72*** |
| G | -0.56*** | -1.05*** | -0.72*** | -1.1*** |
| Control | n/a | -0.49*** | -0.16*** | -0.50*** |
In the table above, ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.