Tech Transfer Central
ARTICLE REPRINT
Experts urge TTO directors to dig deeper for performance measures with more meanin

As their mission expands, TTOs adopt new metrics to showcase a broader view of contributions

This article appeared in the November 2016 issue of Technology Transfer Tactics. Click here for a free sample issue or click here to subscribe.

Don't forget to sign up for Tech Transfer eNews, our free email newsletter filled with helpful tips, industry news, special reports, and key legal and regulatory updates, sent to your inbox every Wednesday!

You'll also receive info on upcoming webinars and other related products.

Performance metrics have long been a topic of hot debate among technology transfer professionals as TTOs try to find new and better ways to document their value. While TTOs still tally things like disclosures, patents and licenses, many offices are now going well beyond these traditional data points, in some cases utilizing more sophisticated approaches that aim to drive certain behaviors and ultimately capture the true contributions of a hard-working office. However, changes in this area are not just designed to do a better job of documenting productivity. They also reflect an expanding “to do” list, handed down to TTOs by states, communities and the institutions they serve.

The model for TTOs has transformed from what it was back in the early-to-late 1990s when licensing officers were primarily focused on filing patients and getting things licensed in a few traditional areas, explains Ragan Robertson, a technology transfer principal at the University of California at Los Angeles and chair of the licensing activity survey committee at the Association of University Technology Managers (AUTM). “Technology transfer organizations have had to change substantially from that model in order to become more integrated in how universities are acting,” he says. “Universities have become more translational-focused and more collaborative-focused … so our TTOs need to be encompassing this market change, and we need to be looking for other ways of expanding how TTOs can interact with universities and … the ecosystem.”

Despite these changes, the things that TTOs typically measure still stem from the older TTO model from the 1990s, and are reflected in the U.S. licensing survey that AUTM puts out every year. But these numbers definitely don’t tell the whole story, observes Robertson.

For instance, universities and their TTOs are now being asked to make more of an impact on regional economic development, but precisely how a given institution can optimally make a difference in this area can vary quite a bit depending on the region, the characteristics of the institution, and the demands from stakeholders. As a result, the specific performance metrics used by TTOs need to be adjusted accordingly.

However, there is no question that the push for economic impact has focused a much bigger spotlight on start-ups — an area where TTOs can make a significant contribution. “The U.S. economy has shown time and time again that small business is where jobs are created more often than with any of the big giant companies, and so from a TTO’s perspective, we can really affect what can happen by helping those start-ups,” explains Robertson.

That still begs the question, how do you measure the level of support that is being provided to start-ups? “I think TTOs are looking at this and saying you know what? We need to have a bigger footprint within the world where money exists. And so a lot more TTOs are looking at building those relationships with angel investors and venture capitalists,” observes Robertson. “One metric that is becoming more apparent is how many connections you have. That is an arbitrary number a lot of the time, but it can help identify that network.”

Some TTOs are measuring the number of connections they have as well as the number of interactions they have had with these key contacts in a given period of time, notes Robertson. “Having those relationships can go very far,” he says. “It is something that is becoming vital for TTOs.”

Dig deeper for meaning

While universities usually do a simple tally of the number of start-ups they help to launch in a given year, most technology transfer professionals now recognize this number is not very revealing, given that many start-ups do not survive for long. “We as an industry need to look at how a start-up is counted … and have a better definition of what a start-up is,” notes Robertson. “My personal opinion is [that a start-up should be counted] when it hits a certain level of investment.”

When someone who is not a founder or an affiliated researcher provides investment, that is when you can begin to discern whether a new enterprise has legs, suggests Robertson. “That doesn’t mean, though, that we shouldn’t continue work with these more nascent start-up entities because they are the ones that are going to turn into [companies] that have firmer foundations [later on], so you may need to have some sort of pre-startup number,” he adds.

Matt McNair, vice president in charge of economic and corporate engagement at Ohio State University (OSU) in Columbus, notes that OSU is doing exactly what Robertson suggests, tracking start-ups but also tracking their success in terms of the level of investment these companies have attracted. However, OSU is also going a few steps further.

“Over the past few years we’ve developed a full funding continuum (8 funds total) to support the development of concepts from early stage to end-product or company,” notes McNair. “This has the effect of both supporting our researchers through the process and reducing risk for those interested in investing in or licensing these technologies. It also makes the lives of our start-up companies easier if the development time is lessened.”

All of this activity is associated with measurable data points that have real resonance with stakeholders. “Currently Ohio State’s portfolio has attracted more than $65 million in funding,” says McNair. “As a result … we’ve also created over 105 jobs. So start-ups, the capital raised, and job creation are all metrics that play a major role in economic development.”

Identify metrics that resonate

In fact, there are few metrics that resonate quite as well as jobs numbers to certain stakeholders, according to Robertson. “That impact alone is [among the] things that all of a sudden politicians are paying attention to,” he says. “If universities are able to foster relationships with their start-ups and give those kinds of numbers — especially about employment — all of a sudden a lot more people become interested.”

Yes, keeping a tally on jobs created takes a lot of work, and it requires TTOs to continually nurture the relationships they have with start-ups over a prolonged period, acknowledges Robertson. “At the end of the day, though, it is something that we have to do,” he says.

Other numbers that can give a more complete picture on start-ups include looking at how many of these companies have actually taken a product or service to market, notes Becky Stoughton, vice president at Fuentek, an Apex, NC-based consulting firm that is focused on advancing innovations. “Then track the sales and revenues at the company regardless of whether they come from licensed products or not,” she says. “Once you have started a company, yes, one of the benefits to the TTO is hopefully at some point you will recoup some licensing revenue or equity depending on how the deal was structured…. But if you are doing [the start-up] for economic development reasons, simply the fact that the company is on the market, is selling and is getting revenues are important things to track.”

Richard Cox, director of the Office of Technology Transfer at the University of Notre Dame, cautions that doing too many start-ups can be problematic, and that benchmarking your own institution’s performance on start-ups against the AUTM licensing survey figures provides important context. For example, a few years ago Cox noticed that Notre Dame seemed to be producing an outsized number of start-ups as a percentage of their licenses. “We were doing 20% to 25% or in some cases 30% to 50% [of our licenses] as start-ups in one year,” recalls Cox.

The TTO administrators happily touted the high number of start-ups at a Board of Trustees meeting and didn’t get the reaction they expected. “One of the trustees said that maybe we were putting an overemphasis on start-ups,” explains Cox. “The trustee pointed out that by our own admission start-ups take longer to get done, they are harder to do, they produce less income and have a higher failure rate. That gave us pause, and we had to step back and think about that,” he says.

Rather than focusing on start-ups to demonstrate impact on economic development, Cox has begun to look at the TTO’s performance against a peer group of institutions that are of similar size and spend similar amounts on research. He has eliminated medical schools from the mix, but includes both public and private institutions. Then he compares the peer group’s performance against Notre Dame on two key measures: conversion of research dollars into invention disclosures and conversion of disclosures into licenses.

“It drives our behavior in the office because we are not yet in the top decile [on performance],” explains Cox. “We may be a little bit above average, but we are pretty much right in the middle, and so then it becomes incumbent on us, if we are going to reach our goal [of being a top performer], to do more engagement with faculty for invention mining.”

Document non-traditional contributions

Frontline tech transfer personnel have long complained that traditional metrics fail to capture much of the value they bring to the table for their institutions. To correct this problem, many offices have developed data points that better reflect the expanding role that TTOs are being asked to play. “We recognized that technology commercialization was just a single touchpoint for companies and that our technology licensing managers are adding value to many relationships which result in quite a few successes for OSU in general that were not within the conventional wheelhouse of a TTO,” explains McNair.

To further enhance these relationships, OSU has established a corporate engagement office that, among other things, serves to reduce the complexity of doing business with a large research institution. But OSU is also working to establish metrics that capture the work and the benefits associated with maintaining these relationships.

For instance, in addition to the traditional metrics that virtually every TTO tracks, McNair notes that OSU is building out its metrics to look at impact with respect to:

  • Student internships and full-time employment placement;
  • Sponsored research dollars;
  • Corporate training and continuing education;
  • Affinity contracts;
  • Corporate philanthropy.

Stoughton is seeing many universities moving in a similar direction as OSU, specifically tracking their TTO’s contributions in areas that go beyond commercialization and licensing. For instance, they’re looking at contributions to sponsored research funding, government grants and even improved faculty recruiting and retention. “Those are some of the trends I have seen starting to take hold, and I think they are very positive trends,” she observes.

Consider processing times

Even on the traditional metrics, though, Stoughton observes that there are ways to extract more meaningful data. For example, while TTOs have tracked the number of disclosures for a long time, they could be putting processes in place to give these numbers much more context. “For instance, you could normalize the outright [number of disclosures] against research expenditures … and then track growth over time,” she explains

Further, when it comes to looking at office productivity, TTOs could be doing a much better job tracking processing times. For example, Stoughton notes that TTOs could be tracking how long an invention disclosure sits in the office before a disposition decision is made. “That is one metric that in my experience very few people try to get any sense of — perhaps because they wouldn’t like the answer,” she says.

Offices could also look at the time period between when disposition decisions are made and when disclosures are actually posted on whatever websites the TTO uses to disseminate this information, Stoughton suggests. “You would be amazed how long it takes some people to take that very simple step,” she says. “Most of these processing metrics are not ones that you would publish or communicate to stakeholders, but you could certainly look at them internally to ensure that you operation is running a well-oiled machine.”

Stoughton advocates for being very strategic in everything a TTO does, and then capturing, analyzing and communicating metrics that showcase these activities, but she believes this type of approach is underappreciated. “One of the perennial situations in tech transfer is the lack of recognition of the offices, and I think often times we bring that on ourselves,” she says. “Some people are better at it than others, but there certainly are a lot of organizations that still don’t do the best job of tooting their own horn. It is partly because the value of it isn’t yet recognized, and because that is the case, they are not devoting the resources to it.”

Scrutinize licensing, patent value

One trend that Ragan finds intriguing is the way some TTOs are giving a score to every one of their licenses based on value, and then going back to re-evaluate these scores years later to see how their valuations held up. “They are keeping track of that to get a better sense of how they are evaluating the technology when it is first licensed,” he says. “This is kind of a big deal for TTOs solely for the fact that our technology is so nascent and so leading edge that a lot of times it is very difficult to capture what field and what market [it encompasses].”

Further, in recent years there has been a lot of discussion among TTO managers about coming up with ways to measure patent quality. This is partly because since the passage of the America Invents Act, university patents have come under increased scrutiny with the advent of inter partes review (IPR), explains Dipanjan Nag, the associate vice president of technology commercialization at OSU. “More than 90% of claims have been invalidated by IPRs instituted against university patients,” he says. “It is critical for universities to evaluate the quality of patents and not just spend blindly on prosecution.”

Nag notes that there are several commercially available options on the market that can be used to examine patent quality metrics using various analytics, and there are also indices that can be used to score overall patent quality. “Simple metrics related to quality patents include but are not limited to the number of claims in a patent, the number of words in a patent claim, and the number of forward citations,” he says.

“Tracking law firm performance for the overall portfolio is also critical to ensure effective and smooth prosecution,” observes Nag. He suggests looking at pendency and the number of requests for continued examination as well as rejections to track the quality of prosecution.

“Patents are of huge importance as they are one of the TTO’s primary products to sell,” stresses Nag. “Patents are perhaps the most valuable asset, yet strangely enough, we do not put a value on our patents. Patents might be a depreciating asset class, but it is important to measure the overall value of all patents and to have them on the balance sheet.”

Contact Cox at rcox4@nd.edu, McNair at mcnair.25@osu.edu, Nag at nag.18@osu.edu, Robertson at ragan.robertson@research.ucla.edu, and Stoughton rstoughton@fuentek.com. 


About Technology Transfer Tactics monthly newsletter...

Find more articles like this one when you subscribe to Technology Transfer Tactics monthly newsletter. Sign up today and get immediate access to our Subscriber-Only Online Resource Center, which includes the entire archive of TTT back issues (since 2007), as well as our treasury of industry research reports, legal opinions, sample forms and contracts, government documents and more.