Blue sky thinking at DVSA on earned recognition

Earned recognition is a flagship DVSA scheme designed to drastically reduce the time wasted checking up on compliant operators in order to free up limited enforcement resources to improve road safety by focusing on the seriously and serially non-compliant.

The aim is for the very best operators to self-certify their compliance and notify the DVSA of any non-compliances outside set key performance indicators, which DVSA managers will then work with the operator to correct. To qualify for earned recognition they must to demonstrate by means of an audit they have robust and validated electronic transport management systems.

DVSA enforcement policy manager Dave Wood (pictured) is the man in charge of earned recognition, which was launched as a pilot scheme involving 26 operators – 18 in the freight sector - with 100 operator licences and 6,000 vehicles in January 2018.

The DVSA categorises operators into exemplar, compliant, mostly satisfactory, non-compliant and serially non-compliant. Naturally earned recognition is aimed squarely at the exemplar and compliant operators, where enforcement can be done remotely, with the middle ground covered by inspections and visits, while the non-compliant will be targeted with increasingly disruptive enforcement activity designed to push them into compliance or out of the industry.

Vision

“Our vision is to deal with these various sectors of the industry in a different way,” says Wood. “We have set up an office looking at remote enforcement products and that is where earned recognition is going to sit.

“If we've got indication an operator might be moving from compliant into non-compliant, we can perhaps make an intervention, not necessarily do a visit, but it could be a remote enforcement activity such as writing to them asking for more information. A lot of the time, that's enough to put them on the right path.”

This approach of educating operators and helping them do the right thing rather than simply punishing minor non-compliance is an important cultural shift at the DVSA, and is why so much time and effort has been put into earned recognition.

It has been partly been forced on the Agency by the lack of resources needed to actively regulate all 73,000 UK O-licence holders. Remote enforcement frees up these scarce resources to target the real rogue operators that put the public at serious risk of death and injury through badly maintained vehicles and excessively tired drivers.

“That's a lot more efficient than having somebody turn up at the operator and completely disrupt the business for a few days,” says Wood. “Obviously, the opposite to that is being completely disruptive, highly targeted on the really bad guys who we want to either get out of the industry or get to a point where they can't possibly operate like that and force them to be compliant.”

While the entire O-licence process is largely based on the trust placed in operators to abide by the terms of their licence, earned recognition takes this trust to a higher level. Operators in the scheme will not be entirely outside the regulatory regime but the DVSA is promising significantly reduced enforcement so it is important the process of getting and maintaining earned recognition status is robust and transparent.

“We will do an application review and there are certain criteria that we're looking for,” says Wood. “There must have been no regulatory action from the traffic commissioner above a warning for the past two years.

It isn’t a precondition that their OCRS [Operator Compliance Risk Score] is all green but compliance history is looked at. If an operator is in the red, then we'd look to see why they were red. OCRS is a risk-rating system and we use it that way as part of the application review. Have there been any bad encounters? Have they had a prohibition? Have they had offenses? If they have, we will look into that a bit more.”

OCRS was amended in 2017 to reduce the number of operators in the grey or unrated band.

“Grey is where there have been no encounters at all,” explains Wood. “Last year we incorporated the two scores for Traffic and Roadworthiness together to drastically reduce the number of greys.

“The operator gets a combined score and that's what we use for targeting. They can still see both types on their reports so they can understand what they are getting from each type of encounter.”

Most operators would regard a green OCRS as a good indicator that they are a legally compliant operator, so why should they go to the extra effort of voluntarily going for earned recognition, which will be colour coded blue to mark it apart from OCRS green?

“It has been one of the challenges because operators think if they are green everything is fantastic,” admits Wood. “OCRS is a fairly basic score, as it is only looking at DVSA’s historical encounters with the operator. We are looking at changing how we deal with OCRS green in the future.”

Once the operator passes the initial application review, they will be invited to seek an audit from one of the DVSA’s approved third party auditors, a list of which can be found on the DVSA website. At time of writing in January 2018, there were 12 approved auditors including the FTA, RHA and CILT. While entry into earned recognition is free, the auditors can set their own fees, so shop around.

The audit will be against a standard written by the DVSA in association with industry bodies and the traffic commissioners.

“We got the industry together and wrote a set of standards that we said ‘this is what we think good looks like’,” says Wood. “Then there's a code of practice for an auditor to make sure we're getting a consistent quality audit. We'll sample the audits as they're coming in and do an annual check with the auditor as well. Anyone can apply and the price will be commercially driven.”

Once within the scheme, operators must pay for re-audits every two years except where they have been given a concession for maintenance systems that do not fully comply with the KPI reporting requirements; then the maintenance module must be audited every year.

Operators with a clean bill of health may be allowed to extend or self-assess audits in future.

Evolving product

“If monitoring is 100% clear for two years, they may be able to extend that audit or maybe even have a self-audit, but that isn’t signed off at the moment,” says Wood. “It's an evolving product.”

The consequences of failing an audit or reporting too many monthly non-compliances are clear.

“If an operator comes out of earned recognition, they just drop into the normal enforcement regime,” says Wood. “There is an exit. That's got to be there to maintain the integrity of the scheme.

“If an operator decided to fraudulently produce the data, it wouldn't be just ‘exit the scheme. It would be exit the scheme, and we'd prosecute you. We would hope that wouldn't happen.”

So after all the hype and rumour, what does an operator actually have to do to move up from merely green to the heights of blue?

“There have been a lot of misconceptions around this,” says Wood. “Initially people were saying we wanted to have access to the operator's data and log onto their systems. When we looked at the feasibility of that, it was just completely impractical. It wouldn't work.

“The key things they do need to have are electronic systems to be able to report key performance indicators for both the driver and vehicle activities.”

“How it works is that driver activities come into the operator's system where they are analysed and the KPIs are monitored. Vehicles are the same but this is the tricky bit. One reason the launch has been delayed was getting all the different maintenance systems to talk to one other, report on one dashboard and give the operator one view of the whole fleet.

“We are making really good progress on this now but if you've got a massive fleet, then it becomes a real challenge. This is why we've made some concessions.”

The problem with electronic systems is the old adage ‘garbage in, garbage out’. How can the DVSA be sure that those nice clean compliance reports it receives each month are an accurate reflection of day to day reality?

“This is one of the things we audit,” says Wood. “An operator, if they were so-minded, could corrupt the data on their system. As part of the validation process we will take raw data from the operator's application. We'll run it through our own analysis, and we'll check against what is coming off their system.

“We validate the system to make sure we're happy it's reporting the KPIs correctly. What it doesn't do is stop the driver deciding to put a magnet on the tachograph. That's where the audit of the operator systems comes in to establish if there are problems with the drivers. Nothing is infallible but it's as much as we can do.”

In arrears

The driver and maintenance systems must be set up to report to DVSA every four weeks in arrears. “They've got chance for their systems to catch up if there are records that it needs to bring up, particularly maintenance,” says Wood. “They then measure the KPIs and if there are no problems, we just get two monthly blank emails from the driver and maintenance systems, basically a nil return, just to make sure they are still connected.

“If we don't get an email, then we go back to the operator to check if something's gone wrong with the system. Or we could get a trigger email, flagging up that the operator has dropped below a certain threshold of one of the KPIs. We don't get the data. All we get is an email which says this KPI has been triggered.”

The operator should be aware that they have missed a KPI as they have the data four weeks before the DVSA, so it should be no surprise that the DVSA will be in touch to find out what has gone wrong.

“They'll say ‘we've triggered and we know what it is. We've put X, Y, and Z in place’. We'll just check that and make sure that we're happy with what they're doing. That's how it works. To reiterate, we don't have access to the data. All the data is held in their systems. We have no access to it.”

The KPIs for drivers and maintenance have been hammered out in extensive consultation with the industry and trade bodies, and set a high benchmark, as you would expect for exemplar operators.

The bands are aligned to the fixed penalty system so the higher the band, the more serious the offence, which is why the permitted percentage failure rate is reduced for the higher bands. This raises the tough issue of when will a ‘KPI failure’ be deemed serious enough to merit a report to the traffic commissioner rather than just a chat with the operator?

“It was quite a difficult process, setting the KPIs,” says Wood. “Why don’t we just report everything to the traffic commissioner? That's what we used to do in the days when we would go in, investigate an operator, find all the offences and put a report together for the traffic commissioner.

“This is now accepting there will be a level of infringements going on and the operator needs to manage that. With the best will in the world you will never have 100% compliance all the time. What this has done is given an allowance to the operator to say ‘yes, we recognise that things will happen, but you've got to be able to manage that’.”

The KPIs have been set as an ‘good operator’ industry average after analysing offending rates is various sectors of the transport industry, and in future Wood would like to see the allowed failure rates come down.

“As we prove the concept, if everybody is managing at 4% overall offence rate, then let's challenge that because we don't want people to offend,” he says, “but we understand that at the minute, it's not realistic to say there's 100% compliance.”

While it is relatively easy to check that drivers’ hours compliance electronic reporting accurately reflects reality just by pulling in a sample of tacho card downloads, making sure that vehicle maintenance reports are true and accurate is more difficult. Just because a workshop has reported the completion of a six-weekly periodic maintenance inspection does not mean the vehicle actually even turned up.

“KPIs will do so much for us, but the audit will make that check,” says Wood. “As much as we can do, we’re trying to keep this as robust as we can. With the full electronic system, because it's captured there and then, it’s imported onto the record as they're doing it, there are a lot fewer data entry errors and missing records.

“Operators do not lose earned recognition just because of the KPI trigger. They get a tolerance of 2% so if a vehicle misses a safety inspection, that would trigger a KPI and we will then have that conversation with the operator.

“We're not asking for any more than the traffic commissioner is expecting. If we set that KPI at 80%, it makes it easy for them to comply, but I'm sure the traffic commissioners won't be very happy if we did that.”

A big issue with maintenance has long been the effectiveness of the drivers’ daily walk around check and defect report. Traffic commissioners often find during public inquiries walk around checks reporting nil defects for weeks on end on trucks which subsequently are found at the six-weekly PMI to have numerous faults.

“The operator should have a system in place to make sure the drivers are trained and that they do the walk around checks. They should check the maintenance records to make sure that they're picking up on driver defects,” says Wood. “This is what we expect the operator to be doing and that's why that audit is so important.

“It is, we feel, as robust as we can make it while being practically deliverable as well.”

MOT pass rates

The 95% first time MOT pass rate is also a stretch, as this is the initial pass rate not the final rate after rectification of minor defects. Wood says that 62% of operators are already meeting that KPI, based on DVSA analysis.

The electronic systems used to report on earned recognition must also give the operator a yellow alert that a KPI is starting to drift before the four-weekly email to DVSA.

“This helps them manage the business,” says Wood. “They will get a yellow alert if one of the KPIs has started to come out of the tolerance. They can get this as quick as they like. This can be weekly for them so they have a very early view of what's going on. This is one of the benefits that the operators have now fed back to us, that this is really helping them get a good view of their whole fleet, and to be able to target where they've got a problem very quickly.”

The amber alert will flag where a KPI is between 1% and 1.9% out of tolerance. When it hits 2% this produces a trigger that is emailed to the DVSA.

“We will then have a conversation with the operator,” says Wood. “One of the key things around this is building a different relationship with an operator where there's this element of trust.

This is quite a challenge for us, because of our status as the enforcement agency. A lot of operators have said ‘as soon as we've triggered an infringement, we'll be straight to the traffic commissioner’. That's not the case.

“Essentially these are really good operators, running at a very strict tolerance on these KPIs. So let's have a look, see what the problem is and then resolve it. But of course, if they keep triggering every month, and we're spending quite a bit of time on them, then earned recognition isn't working and they come out into normal enforcement.”

The whole premise of earned recognition is that it frees the DVSA up from checking up on compliant operators to focus on the serially non-compliant. So if it turns out more resource is being allocated to helping earned recognition operators correct very minor problems it will have failed.

“One of the things we've been doing in the pilot is seeing how many triggers we get,” says Wood. “That was a concern for us, obviously because if every operator is triggering every month, then it's wasting time. If those KPIs aren't working for us, then we'd have to look at them. But as it's working so far, we're not getting that many triggers.”

In the pilot phase before the launch in January two measures were dropped to make the system workable.

“One was missing mileage and the other was repeat offenders,” says Wood. “We tried to capture where a driver was continually offending month after month. Although that's still reported for the operator, because it's information the operator needs to deal with, it was triggering false alerts.”

Equally, enough operators must be included in earned recognition to justify the effort and expense in setting up the system. At the launch of the pilot in January, 26 operators with 6,000 vehicles were included.

“Our staff know who they are and treat them differently, as far as roadside inspections and so on,” says Wood. “They could still get stopped and checked. If we see something dangerous such as the driver on a mobile phone then we'll stop it, it doesn't matter if they're in earned recognition.

“It's all looking positive to go live in the spring. If we get 10% of the UK fleet included, then that starts making quite a big difference for us, as far as releasing resources, and that is what we have predicted for the first year. That would start making quite a significant contribution to diverting resource into targeting the bad guys.”

Traffic commissioners

Earned recognition could not work without the support of the traffic commissioners, as they have to accept that the DVSA is making a judgement call on which offences to report for operators within the scheme.

“The traffic commissioners have been fully engaged with how we build the product,” says Wood. “We still have our legal obligations if we find most serious offences. What we want to do is make sure that these are the exemplar operators that really do care about compliance. It is going to be aspirational for a lot of operators. As it rolls out, and people see that it's working, the confidence will come in the industry to want to be in it.”