Emerging Legal Technology Forum 2017: Training, Metrics, ROI and Looking for the Viagra Effect

Topics: Business Development & Marketing Blog Posts, Efficiency, Emerging Legal Technology Forum, Law Firms, Legal Executive Events, Legal Innovation, Midsize Law Firms Blog Posts, Talent Development, Thomson Reuters

emerging legal technology forum

TORONTO — Once a legal department or law firm has decided to invest in a significant piece of legal technology, determining the return on investment (ROI) and applied benefits can be daunting, but convincing users it can improve their life and getting good adoption can be even more so.

During a panel called Chain Reaction: Managing the Strategic Implementation of Legal Technology at the Thomson Reuters’ Emerging Legal Technology Forum 2017 in Toronto yesterday, a panel of legal technology and knowledge management professionals provided their tips for successful implementation.

The first step to success and measuring true ROI starts with training, says Dera Nevin, eDiscovery Counsel and Director of eDiscovery services at Proskauer Rose. For example, Nevin said when rolling out an e-discovery application, she places all users, even super users, into the lowest level of access rights. “They will say, ‘But I need to mass tag documents’ and I will say, ‘That’s great, I have a training program for you and when you complete my training program I will give you those elevated rights.’”

She has a couple of reason for doing it this way, Nevin explained. “One is because it forces people into training but also then I have the confidence that those people understand the technology,” she said. “I basically tell people I will give you the rope to hang yourself when I know that you won’t do that.”

Nevin said it can be difficult to get alignment with Human Resources on training of new technology, so that’s why she holds the “gate keeper” role. “If you want to be able to mass tag you have to go through my training program and if you can’t mass tag you will be less effective as an associate, less efficient and less accurate,” she noted. “I will be able to show you how you will be less accurate so you will want to get on board.”

In order to encourage users to go beyond what she calls the “splash pad” of the final launch of an implementation she also articulates key performance indicators (KPIs) at the beginning that are tied to return on investment — and then measures them. “One thing I do is look at how many associates have gone from Level 1 to Level 2, then how many associates have gone to Level 3? How many questions do we get? What is the nature of the help desk calls we’re getting to show people the change over time?” By measuring KPIs you can then articulate the ROI, Nevin added.

Emerging legal technology forum

Kimberly Stein, Thomson Reuters Legal; Ginevra Saylor, Dentons; James Anok, Fireman & Company; Dera Nevin, Proskauer Rose; Laurence Liss, Blank Rome; and Natalie Munroe, Osler Hoskin & Harcourt.

She also tries to have someone on the deployment team “looking for the Viagra effect” — the inevitable someone who is the person saying ‘This technology is never going to work’. This can allow you to find the naysayers but also find what the technology may also be useful for, because often technology is pitched at user groups as intended to do X but is way better at doing Y.

“It’s just as Viagra started out as heart medication but didn’t work so well, but really worked well at something else,” she said. “I always have somebody looking for how are people are actually using it; and maybe there is an off-label use case that is better and that we should develop training for.” Similarly, it’s also important to look for blind spots — who is talking trash about the system — and really listen to what the problem with the deployment has been, she added.

Natalie Munroe, head of Osler Works-Transactional at Osler, Hoskin and Harcourt, agreed, adding that identifying a firm’s “super users” of technology is helpful in getting implementation off the ground.

James Anok, Senior Consultant at Fireman & Company, explained that the measure of project success or failure is understanding at the outset that you have to develop metrics to measure success. “You have to define measures of success and thresholds that you have to meet this within a certain time otherwise why should I use this? What is my driver? If not measured against it why should it be adopted?” he asked.

Ginevra Saylor, National Director of Knowledge Management at Dentons in Toronto, also spoke about the importance of being able to tell stories about how someone’s work is better and how the client is happier — all as a direct result of using the technology. And while metrics and KPIs are important, expressions of satisfaction with a tech rollout also can come in the form of more simple human responses. “The one I like is when people have said to me: ‘I don’t hate my job anymore’,” Saylor said.

Anok agreed, saying that ROI can be measured in many ways other than with numbers, such as satisfaction internally and with clients. “Are you able to retain people because they like where they work and are comfortable with the technology?” he asked. “It’s not just about can we make it easier, but easier so you can focus on things that are more interesting — take away the mundane so you can focus on higher value things.”