Tech Transfer eNews Blog
Industry-Sponsored Research Management sample issue

How to use peer review to guide TTO performance improvement

By Jesse Schwartz
Published: May 17th, 2017

We’re all familiar with the warning: Be careful what you wish for. And that can certainly be true if you seek to learn the “absolute truth” about your TTO’s performance. But that is the aim of a peer review assessment process discussed in a panel session at the AUTM 2017 Annual Meeting last month.

Peer review is “typically absolute,” observed Ashley J. Stevens, D.Phil. (Oxon), CLP, RTTP, principal of Focus IP Group. In other words, he explained, it asks the question: “Is this office doing a good job?” as opposed to: “Are we doing the best we can with the [stuff] we get?” In short, it “looks at every aspect of an office’s performance.”

Why might a peer review be considered? There could be discontent in the ecosystem, said Stevens, or there might be a new director of research (“probably a very bad sign,” he observed), or it could be a routine, system-wide procedure. For example, his fellow panelist, Ruth Herzog, PhD, of the German Cancer Research Center (DKFZ), noted that her institution conducts a peer review every five years.

Stevens pointed out that there are three options for the process: self-evaluation, local evaluation, and international evaluation.

A self-evaluation is often requested by management “to assure the TTO is state of the art, or if it needs to change,” Stevens said. These are led by the director, and can be “incredibly difficult,” he declared, because from the TTO’s point of view “if we thought things could be done any better we’d have already changed it.” What’s more, he added, they probably don’t recommend anything radical, except more resources, which management “really doesn’t want to hear.” In some cases, an internal peer review could be done pre-emptively to identify systemic problems.

In a local evaluation, management retains local consultants, with the benefit of providing results in a regional context. A similar evaluation done with international consultants loses the local context but gains a global one, so your choice should be based on which peers you wish to be compared against.

Stevens emphasized the importance of benchmarking as part of the peer review process. You need “a standardized set of data, or the willingness of your peer group to provide data not otherwise collected,” he explained. “You must benchmark against comparable institutions.”

Stevens suggested benchmarking in “3D” – disclosures, deals, and dollars. The peer group should be comparable in research volume, type of institution (public, private, teaching hospital) and the type of research conducted. The ecosystems should also be comparable – i.e., innovation hub, rust belt, cornfields, emerging economy.

It’s also important, he added, to benchmark variables that are under your control, such as disclosures per million dollars of research funding, increases year on year, patents, reimbursement, expenditures per disclosure, and expenditures per issued patent. The same applies to staff — for example, disclosures per FTE and caseload.

Other variables under your control, Stevens continued, include number of deals executed and license success rate, total number of active licenses, and startups (number, number based locally, number securing funding, number with full-time employees).

Just as important, warned Stevens, is that you “avoid variables that are not under your control like the plague.”

Take, for example, income. “Today’s income generally results from deals done by your predecessor — or theirs,” he noted. The same goes for “big hits.” These are random, said Stevens, and if you are tempted to take credit, you should remember that patents expire. “An amazing number of directors retire the year the big patent expires,” he observed.

“What you do control is current deal value – up-fronts, milestones, equity,” he concluded. A detailed article on conducting peer review assessments appears in the April issue of Technology Transfer Tactics. To subscribe and access the full article, as well as the publication’s 10-year archive of proven best practices and success strategies for TTOs, CLICK HERE.


Posted under: Tech Transfer e-News

Twitter Facebook Linkedin Pinterest Email

No Comments so far ↓

There are no comments yet...Kick things off by filling out the form below.

Leave a Comment