Lies, damn lies and ... metrics


The word “metrics” is so commonly used in modern business life that nobody thinks they need a definition of the term. It hardly occurred to me to look it up before starting this blog. But we probably ought to, because there's a fair bit of groupthink on this topic. If you open the Oxford English Dictionary, once you’ve made it past the definition of the “metric system”, which remains a surprisingly lively political topic, you'll find “metrics (in business)” defined as: “a set of figures or statistics that measure results”. That’s short, but it will probably do.

After discussing metrics with a large number of senior legal colleagues over the past year, I’ve concluded that metrics are widely distrusted by the legal profession. The topic certainly polarizes opinions. The most common charge made against metrics is that they provide a reductivist view of an activity which involves complex interactions that aren’t really capable of or susceptible to numerical measurement, such as the giving of legal advice.

Lawyers tend to have an unhappy relationship with metrics from the day they start in practice. The metrics demanded by the firm’s Managing Partner usually include: “billable hours” - the awful symbol of the grindstone to which lawyers are tied; “profits per equity partner” - for junior lawyers just a measure of how much money someone else is making; “aged receivables” - which translates into painful discussions with the client on when they are finally going to pay that bill. Clients are no better, because they demand metrics too. Litigators groan when they hear the inevitable question “what are my percentage chances of winning?” Almost all lawyers groan when asked the question “how much is this going to cost?” Numbers, numbers. It’s all so tiresome.

For the past three months, I've occupied a line management rather than a legal role, as acting Chief Executive Officer of a publicly listed company with 4,200 employees, operating in over 100 countries and making hundreds of millions of dollars in annual sales. I hope that explains why this blog has been so quiet. Every two weeks, I’m handed a two inch thick pack containing metrics on our company’s operations. Each business measures different things, but my own pack contains, just as an illustration, very detailed pages on sales deals, regional performance, order intake, revenue, costs, profit, cash collection, receivables, DSO (how fast customers are paying), headcount, employee attrition, utilization of service staff etc. In almost all cases, an important element the metric is how it has changed, its variance, over time. 

It’s hardly controversial to assert that people managing complex situations need data to help them form judgements. The pilot of an aeroplane has a large number of instruments in the cockpit giving him/her data about the aeroplane and its progress. Likewise, any CEO or CFO needs data to help them run a company.

That said, metrics are clearly no substitute for a vision of where you are going, the human instinct on how best to get there and the ability to galvanize large numbers of people to follow you. But, in business management roles, almost everyone believes that metrics and their variance can bring insight. I want to repeat those three words … can bring insight. It’s important to understand that metrics are not, in themselves, insight. But they can facilitate insight. I’ve sat in meetings where people have said “I don’t believe these numbers” or “I don’t know what these numbers are telling us”. I’ve said those words myself. It’s a very important part of scrutinizing all metrics that you ask yourself “do I trust these numbers?” and “what do these numbers actually mean?”

So, when we all know that metrics are so widely used by others, why do lawyers remain suspicious of them? I think the answer lies in a misunderstanding that you most commonly see between accountants and lawyers. Accountants believe that words (for example in a contract) are very precise. They will often seek a definitive legal opinion on their meaning. Lawyers know, from long experience, that words are malleable and can have different meanings. Meanwhile, lawyers are convinced that numbers are very precise and often take them at face value, whereas accountants know that numbers are malleable. You will often see accountants asking questions about the assumptions that underlie a set of numbers.

My own view is that the camp in the legal profession that thinks metrics are reductivist and unhelpful is the same camp that thinks business leaders will fail to question them because they carry some kind of absolute certainty. But, speaking as a business leader, we are not that simple. For example, if you show a business leader a set of metrics from the legal department which demonstrate that customer contracts are taking longer to complete, they will first ask “do I believe these numbers?” and then “what do these numbers mean?” It’s simplistic to assume business leaders will conclude that the meaning of such a metric is that the legal team is being slow with its work. If you want a recent real example of your correspondent interpreting metrics, you can find it on the BBC website here http://www.bbc.co.uk/news/business-17429412

As it happens, my own business doesn’t consume many metrics from the legal department. In any given business, what’s worth measuring is going to depend on various things. Some legal metrics are “internal” and will help the GC run the legal function, but not provide insight to business colleagues. An example might be the time it takes to process a contract into the department’s contract storage mechanism. Other metrics are “external", useful to business colleagues. In a company relying heavily on export licenses, an example might include the measurement of cycle times in applying for and securing those licenses.

I don’t believe that many legal functions have metrics which are genuinely critical for running a business. If they did, legal functions would be generating more metrics. But legal teams could probably generate more useful metrics than business leaders currently realize, if GCs were prepared to offer them. 

My advice to GCs is not to be defensive. You and your business colleagues may not know what a particular metric means. To decide, you’ll need insight. But metrics are a powerful tool in running any business and the legal function is no exception.

Comments

  1. Nicely done Tom!

    Metrics without context are a waste of time.

    Metrics can be helpful when they relate to what you want to accomplish and when they measure change or trends. And, perhaps most importantly, you must take the next step and use them to accomplish something--change, improvement or whatever.

    ReplyDelete
  2. A very interesting blog Tom that gives much food for thought. It covers (at least) two distinct areas where metrics are appropriate: managing performance of a team / the business (usually historic); and decision making on cases where there is uncertainty about the future. This comment refers to the former.

    I have worked with lawyers over many years (and was married to one for 20) and in my experience they fall into two types as far as numbers are concerned. There are those who come from a scientific and numeric background for whom metrics are a natural part of life. There are others I meet however, who visibly shudder when they discover I was originally a mathematician. How do we help this latter group?

    There is a fascinating book called "Why children fail" by John Holt that explains how the fear of failure gets in the way of problem solving. When I was a Maths teacher my main task was to create an environment where students felt free to "get it wrong" so they could work their way to a solution. People need to be free to focus on the problem, rather than worry about "the right answer", especially when it comes to metrics. It's ok to play with metrics, look at them upside down and backwards: it's not a question of "right" (red tick) or "wrong" (red cross).

    So what I would recommend is this. Concentrate firstly on "what" it is you need to know in order to manage or improve your business, then the rest will follow. Metrics become merely a useful tool rather than an isolated end in themselves

    Examples include:

    For different case types, what is typical and what is atypical? For instance what is the average elapsed time from start to finish and how much effort is applied? How can we identify exceptions and manage these?

    How are we doing as far as clients are concerned? What would help them keep track of where we are on their cases?

    Some parts of the team seem to be working really hard and under pressure, whilst others seem to have less to do than this time last year. How can I get a handle on this and balance the workload?

    Once the top down management issues are phrased in these terms, the metrics generally flow and are meaningful. If not perfectly expressed first time no matter: they can be honed and improved over time.

    As for decision making on cases where there is risk, managing likelihood etc, that is a very different question. I will give that some further thought and comment late on that. It's a big subject!

    ReplyDelete
  3. Tom,

    Excellent blog. There's so much to say on this subject!

    Language often gets in the way - metrics, data, management information (MI), reporting ... etc. But whatever you call it, too much of it can also be a problem; providing false 'comfort' (look at all the data we have) but absolutely no insight (what is it telling us?).

    Many organisations we work with are swamped with MI but fail to appreciate two key issue; it's what you do with it that counts and you get what you measure (i.e. be careful what you measure because it drives behaviours).

    We have been surprised by how little real measurement exists in the legal market; whether this is the performance of the in-house function and /or panel arrangements. Measurement is key as long as it is combined with analysis and action.




    T

    ReplyDelete

Post a Comment

Popular posts from this blog

No timesheets! Making the transition from private practice to in-house

If you aren't in on Saturday, don't bother showing up on Sunday

If your GC reports to the CFO, your company doesn't understand what Legal does