Lies, damn lies and ... metrics
The word
“metrics” is so commonly used in modern business life that nobody thinks they need
a definition of the term. It hardly occurred to me to look it up before
starting this blog. But we probably ought to, because there's a fair bit of groupthink on this topic. If you open the Oxford English Dictionary, once
you’ve made it past the definition of the “metric
system”, which remains a surprisingly lively political topic, you'll find “metrics (in business)” defined as: “a set of figures or statistics that measure
results”. That’s short, but it will probably do.
After
discussing metrics with a large number of senior legal colleagues over the past
year, I’ve concluded that metrics are widely distrusted by the legal profession.
The topic certainly polarizes opinions. The most common charge made against
metrics is that they provide a reductivist view of an activity which involves complex
interactions that aren’t really capable of or susceptible to numerical
measurement, such as the giving of legal advice.
Lawyers tend
to have an unhappy relationship with metrics from the day they start in
practice. The metrics demanded by the firm’s Managing Partner usually include:
“billable hours” - the awful symbol of the grindstone to which
lawyers are tied; “profits per equity partner” - for junior lawyers just
a measure of how much money someone else is making; “aged receivables” - which translates into painful discussions with the client on when they are finally going to pay
that bill. Clients are no better, because they demand metrics too. Litigators
groan when they hear the inevitable question “what are my percentage chances of
winning?” Almost all lawyers groan when asked the question “how much is this
going to cost?” Numbers, numbers. It’s all so tiresome.
For the past three months, I've occupied a line management rather than a legal role, as acting Chief Executive Officer of a publicly
listed company with 4,200 employees, operating in over 100 countries and making
hundreds of millions of dollars in annual sales. I hope that explains why this blog has been so quiet. Every two weeks, I’m handed a
two inch thick pack containing metrics on our company’s operations. Each business
measures different things, but my own pack contains, just as an illustration, very detailed pages on sales deals,
regional performance, order intake, revenue, costs, profit, cash collection,
receivables, DSO (how fast customers are paying), headcount, employee attrition,
utilization of service staff etc. In almost all cases, an
important element the metric is how it has changed, its variance, over time.
It’s hardly
controversial to assert that people managing complex situations
need data to help them form judgements. The pilot of an aeroplane has a large
number of instruments in the cockpit giving him/her data about the aeroplane
and its progress. Likewise, any CEO or CFO needs data to help them run a
company.
That said, metrics
are clearly no substitute for a vision of where you are going, the human
instinct on how best to get there and the ability to galvanize large numbers of people to follow you. But, in business management roles, almost
everyone believes that metrics and their variance can bring insight. I
want to repeat those three words … can bring insight. It’s important to
understand that metrics are not, in themselves, insight. But they can facilitate insight. I’ve sat in meetings where people have said “I don’t believe these numbers”
or “I don’t know what these numbers are telling us”. I’ve said those words
myself. It’s a very important part of scrutinizing all metrics that you ask
yourself “do I trust these numbers?” and “what do these numbers actually mean?”
So, when we
all know that metrics are so widely used by others, why do lawyers remain suspicious
of them? I think the answer lies in a misunderstanding that you most commonly
see between accountants and lawyers. Accountants believe that words (for
example in a contract) are very precise. They will often seek a definitive
legal opinion on their meaning. Lawyers know, from long experience, that words
are malleable and can have different meanings. Meanwhile, lawyers are convinced
that numbers are very precise and often take them at face value, whereas
accountants know that numbers are malleable. You will often see accountants asking
questions about the assumptions that underlie a set of numbers.
My own view
is that the camp in the legal profession that thinks metrics are reductivist and
unhelpful is the same camp that thinks business leaders will fail to question them because they carry some kind of absolute certainty. But, speaking as a business leader, we are not that simple. For example, if you show a business leader a set of metrics from the legal department which demonstrate that customer contracts
are taking longer to complete, they will first ask “do I believe these numbers?” and
then “what do these numbers mean?” It’s simplistic to assume business leaders will
conclude that the meaning of such a metric is that the legal team is being slow
with its work. If you want a recent real example of your correspondent interpreting metrics, you can
find it on the BBC website here http://www.bbc.co.uk/news/business-17429412
As it
happens, my own business doesn’t consume many metrics from the legal department. In any given business, what’s worth measuring is going to depend on various things. Some legal metrics are “internal” and will help
the GC run the legal function, but not provide insight to business colleagues. An
example might be the time it takes to process a contract into the department’s
contract storage mechanism. Other metrics are “external", useful to business colleagues. In a company relying heavily on export
licenses, an example might include the measurement of cycle times in
applying for and securing those licenses.
I don’t
believe that many legal functions have metrics which are genuinely critical for
running a business. If they did, legal functions would be generating more
metrics. But legal teams could probably generate more useful metrics
than business leaders currently realize, if GCs were prepared to offer them.
My advice to GCs is not to be defensive. You and your business colleagues may not know what a particular metric means. To decide, you’ll need insight. But metrics are a powerful tool in running any business and the legal function is no exception.
Nicely done Tom!
ReplyDeleteMetrics without context are a waste of time.
Metrics can be helpful when they relate to what you want to accomplish and when they measure change or trends. And, perhaps most importantly, you must take the next step and use them to accomplish something--change, improvement or whatever.
A very interesting blog Tom that gives much food for thought. It covers (at least) two distinct areas where metrics are appropriate: managing performance of a team / the business (usually historic); and decision making on cases where there is uncertainty about the future. This comment refers to the former.
ReplyDeleteI have worked with lawyers over many years (and was married to one for 20) and in my experience they fall into two types as far as numbers are concerned. There are those who come from a scientific and numeric background for whom metrics are a natural part of life. There are others I meet however, who visibly shudder when they discover I was originally a mathematician. How do we help this latter group?
There is a fascinating book called "Why children fail" by John Holt that explains how the fear of failure gets in the way of problem solving. When I was a Maths teacher my main task was to create an environment where students felt free to "get it wrong" so they could work their way to a solution. People need to be free to focus on the problem, rather than worry about "the right answer", especially when it comes to metrics. It's ok to play with metrics, look at them upside down and backwards: it's not a question of "right" (red tick) or "wrong" (red cross).
So what I would recommend is this. Concentrate firstly on "what" it is you need to know in order to manage or improve your business, then the rest will follow. Metrics become merely a useful tool rather than an isolated end in themselves
Examples include:
For different case types, what is typical and what is atypical? For instance what is the average elapsed time from start to finish and how much effort is applied? How can we identify exceptions and manage these?
How are we doing as far as clients are concerned? What would help them keep track of where we are on their cases?
Some parts of the team seem to be working really hard and under pressure, whilst others seem to have less to do than this time last year. How can I get a handle on this and balance the workload?
Once the top down management issues are phrased in these terms, the metrics generally flow and are meaningful. If not perfectly expressed first time no matter: they can be honed and improved over time.
As for decision making on cases where there is risk, managing likelihood etc, that is a very different question. I will give that some further thought and comment late on that. It's a big subject!
Tom,
ReplyDeleteExcellent blog. There's so much to say on this subject!
Language often gets in the way - metrics, data, management information (MI), reporting ... etc. But whatever you call it, too much of it can also be a problem; providing false 'comfort' (look at all the data we have) but absolutely no insight (what is it telling us?).
Many organisations we work with are swamped with MI but fail to appreciate two key issue; it's what you do with it that counts and you get what you measure (i.e. be careful what you measure because it drives behaviours).
We have been surprised by how little real measurement exists in the legal market; whether this is the performance of the in-house function and /or panel arrangements. Measurement is key as long as it is combined with analysis and action.
T