You Want to Measure Quality in Contracts? Without a Style Guide, You’re Nowhere

I noted with interest this post by Ken Grady on Seyfarth Shaw’s Seytlines blog, particularly as last year I did a Q&A with Ken on this blog (here).

Ken’s post is about quality in contracting. He starts by discussing the limitations of determining quality by proxy. As he says, “Trusting the brand, versus trusting metrics that measure desired characteristics, is measuring something by proxy.” According to Ken, the alternative to is to measure quality itself:

I’ll walk through a few quality measures that are easy to introduce when working with documents. These aren’t the only quality measures and I’m not even going to argue they are the best. These are a starting point.

Ken then proposes the following quality measures:

Readability, of the sort measured by, for example, the Flesch-Kincaid test.
Accuracy, namely avoiding typos, incorrect dates, inaccurate cross-references, and the like.
Custom, which appears to relate to setting targets for key points in the contracts process.

Ken continues as follows:

The quality metrics I discuss above are just the beginning. We can add other metrics, such as whether a contract results in a dispute, the severity of the dispute, the cost of a contract over its lifecycle, the number of times a contract must go back and forth between drafters before completion, and so on.

But I suggest that Ken’s measures of quality are simply another way to measure quality by proxy:

As I discussed in this post way back in 2006, using readability tests is a crude way to assess contract prose.
One certainly wants to avoid mistakes, but that part of the process is like running final checks after a car comes off the production line: it ensures that the car conforms to specifications, but it doesn’t measure the quality of the car as designed.
As for failing to meet contract-process deadlines, that’s a symptom of a problem rather than a problem itself. The same goes for any disputes that arise.

The only way to measure the actual quality of contract prose—broadly speaking, not what you say but how you say it—is to compare it to your organization’s style guide. What, you don’t have a style guide? That pretty much guarantees that your contracts contain an inconsistent mishmash of the dysfunctional usages that characterize traditional contract language.

But any old style guide won’t do. For example, a style guide that reflects the dysfunction of tradition contract language wouldn’t be of any use. Furthermore, to be effective, a style guide has to be more than just a few pages—there’s a reason why MSCD is more than 500 pages. But it’s not realistic to expect any organization to create its own comprehensive style guide. That’s why I’ve put together a model “statement of style” (here), which is an example of a short document which an organization can use to say that it’s adopting a style guide based on MSCD.

When it comes to substance, determining quality is tougher. It’s not susceptible to quick checks; there’s no alternative to getting the input of subject-matter experts and working through multiple drafts.

But measuring the quality of contract prose is the place to start. And if you don’t have a style guide for your contract language, you’re not serious about quality.