In analyzing patient recruitment funnels, we at Clara utilize what we call - quite simply - the Clara Funnel Analysis Framework. This framework is made up of 25 metrics that we measure for all of the trials we work with, on a daily basis.
25 might sound like a big number too unwieldy for daily analysis. In reality, the framework is quite intuitive and makes the work of analysis frictionless. This is because the framework, at its heart, is simply the recruitment funnel, the same one that a researcher designed before initiating her trial and knows by the time recruitment comes around like the back of her hand.
And with the framework anchored by the recruitment funnel, we use two sets of companion measures that help us understand how our funnel is performing, as well as two diagnostic measures that allow us to quickly locate any potential problem spots within a funnel.
- Ensure that each step of your recruitment funnel is accounted for
- Measure the conversion rates from one step to the next to analyze daily performance
- Heat maps/user behavior reveals what is happening at the top of your funnel; response times from your team often dictate how the bottom half of your funnel will progress.
- Ensure all costs are captured to uncover where team/budget allocations can be optimized.
The way we break down a recruitment funnel is as follows:
As you can see, the funnel takes us from the moment someone is made aware of a trial (the impression) to the moment that a person enrolls in the trial. By breaking out the funnel step-by-step, a researcher using her companion and diagnostic measure will be equipped to, at any given time, locate exactly where the frictions in her recruitment strategies lie, and how to begin addressing them.
Of course, each trial design is different. It may be that a trial’s screening process requires multiple visits, or that a study can go straight from consent to enrollment, or that some of these steps are ordered differently. Given this reality, it is our recommendation that any trial team sits down to map out exactly what a potential enrollee’s journey from awareness to enrollment may look like, and work to capture every significant step. Breaking out each step as granularly as possible will ensure that the analysis is holistic, and that potential optimization opportunities do not get missed.
That said, let’s talk about how we think about each of the common steps in any recruitment funnel:
Impressions measure how many times your message is received by an audience.
In the case of digital advertising, each instance of an ad unit being shown is considered an impression. One major benefit of using digital advertising is precision; you will know on a day-by-day basis (and even on an hour-by-hour basis) how many impressions you are winning for your message, down to the single digits.
In traditional media - for example, a newspaper or television ad, a billboard, an ad on public transit - impression counts are generally less precise (after all, it’s difficult to truly know exactly how many eyeballs look at a billboard on a given highway); however, any vendor selling these ad units will be able to offer an estimate of expected daily impressions.
We recommend obtaining as accurate an estimate as possible from any third party vendor, be they working in the fields of traditional or digital media. Impression counts build the foundation of funnel analysis within our framework; it's only through ensuring accuracy at the base that we can make reliable sense of what our companion and diagnostic measures are signaling to us.
At Clara, we measure how many people visit a landing page that we have created for a clinical trial. In this digital age, almost all trials work internally or with a third party to create a landing page for their studies. In each of these cases, it is important to measure the number of visits occurring. Fortunately, any IT team will be able to access visit volume data. Our recommendation is that, if you do not already have easy access to your visit data, you work with your vendors or internal teams to receive daily reporting.
Upon a visit, we are effectively being granted our first chance at making a good impression upon a potential enrollee. As such, we think of landing pages as our handshake with our audience. And because our first chance to make a good impression can quickly become the last chance we get if we do not do a great job in engaging the interested party, it is important to know exactly how many people we’re greeting with our landing page, day over day.
A lead is any person who leaves their contact information with us in order to be contacted. This is the first time in our funnel where we see someone who may be interested in the study indicate that interest explicitly. This means that from this point on, we are no longer simply welcoming someone; we are now tasked with making sure our hospitality is up to snuff as we help the leads graduate through the funnel. (More on hospitality in response time.)
In our case, most of the leads come through our landing pages that utilize optimized contact info capture modules. However, it is often the case that leads come to a study team through email, phone call, text, and even faxes, as well as digital landing pages. Be sure to include every lead in this measure, no matter the channel.
This is simply the number of people who complete your pre-screener. A good related measure is the number of people who have received or started your pre-screener.
This is the number of people who have passed your pre-screener.
After this point, every clinical trial design looks unique. It may be the case that a trial has no final screening process, or requires multiple phases of consent. In any case, be sure to break out as granularly as possible every single step that sits between a pre-screened person and the final step of enrollment.
In general, the final steps of any recruitment funnel look something like the following:
This is the number of people who have consented to enrolling in your study.
This is the number of people who pass through final screening for your study.
Finally, This is the number of people who have enrolled into your study.
These measures make up the heart of our daily analysis. By maintaining a good grasp on these fundamentals, any recruitment effort can be optimized over time.
Companion measures: conversion rates
The funnel is the center of our analytical universe. Orbiting this are two sets of companion measures: Conversion rates and costs.
We use these companion measures because raw numbers do a poor job of telling anyone an accurate story. Consider that 100 completed pre-screeners might sound good, until it is revealed that 10,000 leads were used to get to those 100 completions. We therefore need to understand how a cohort is moving from step to step in our funnel.
In other words, we need to measure conversion rates between each step of the funnel. A conversion rate is a measure that tells us what percentage of a group graduate from one step to the next.
The conversion rates we measure are as follows:
It’s worth repeating that each trial design is different, and that the steps in the recruitment funnel can look quite different from one trial to another. Measuring the conversion rates from each and every step of the recruitment funnel will give you a holistic look at exactly how the funnel is performing.
And while conversion rates from, say, consented-to-screened varies too wildly by study design for us to set reliable industry average standards, the first few conversion rates do have industry averages that we use as benchmarks at Clara.
The industry average for this measure is about 1%. At Clara, we use sophisticated targeting in the digital ads that we run for clinical trials, so we aim for a 2-3% target. While aggressive on its face (it is, after all, double to triple the industry average), we’re able to hit the target because of the work we do in ensuring that our ads get shown to the most relevant possible audiences. Put another way, we ensure that the top of our funnel is filled with an already pre-qualified audience. This isn't unique to our team: Any recruitment effort can benefit from the cascading and compounding effects of impression optimization.
The industry average for this measure is about 2%.
Because we use our own landing pages that are optimized from launch to make it easy for anyone to find and drop their contact information, we aim for a 3-4% target internally. Again, this isn't necessarily unique to our capabilities: Any team can optimize this conversion rate by ensuring that a landing page's contact capture module is easy-to-find and intuitive to any visitor.
You can learn more about landing page optimization here, where we look to consumer tech leaders like Airbnb and Uber to draw out tactics that can help any patient recruitment effort.
When it comes to taking a person who has left their contact information to complete a survey, we use an aggressive standard: 10%. This means that we aim to ensure that 10% of all people who have indicated explicit interest will complete our pre-screener survey.
We help ensure this high standard through the use of automated texts, personalized emails, and other participant outreach; we optimize at this point for maximum hospitality, such that every interested person is responded to quickly and with ample information. We also utilize our own HIPAA-compliant surveys that can exist on our landing pages.
So, as it turns out, 10% isn’t too scary a number, as long as we ensure that our pre—screener survey is accessible (in multiple ways), fast to complete, and easily understandable to the would-be participant. Every recruitment effort can improve this conversion rate by focusing on quick responses, optimized pre-screen survey design, and easy survey accessibility.
Companion measures: Costs per
The final set of measures we look to daily are cost per measures. While simple on its face, there are a few nuances that we look to capture in this analysis to uncover potential or pre-existing inefficiencies in a funnel's design.
The costs per we focus on are as follows:
- Cost per Impression
- Cost per Visit
- Cost per Lead
- Cost per Pre-screen
- Cost per Passed
- Cost per Consent
- Cost per Screened
- Cost per Enrolled
Cost per Impression:
This one is pretty simple - it’s simply the total cost spent in spreading your message, divided by the number of impressions (both estimated and measured, across every channel) won through your effort.
One common pitfall here is an over-optimization for cost per impression. There is no shortage of opportunities for a buyer to purchase a big impression package with aggressive, attractive CPMs (cost per mille, or cost per thousand impressions, which is how most people buy and sell ads). However, buying huge reach for low CPMs often backfires down the funnel, when costs begin building up not just monetarily, but in terms of your team’s bandwidth. To make matters worse, these costs flow down the funnel and compound, growing costlier as we move down towards enrollment.
As a quick thought exercise, consider this: A media seller offers your team the opportunity to buy a television ad that can reach an estimated 1M impressions over seven days, for the low, low cost of a $0.05 CPM. This sounds attractive - especially if your targeted digital efforts are running at perhaps $0.45 CPMs - but if that traffic turns out to convert far worse than your better channels, it can become an albatross around your marketing budget. Furthermore, if your team is spending effort and time attempting to graduate this glut of unqualified traffic from one step to the next, costs at each step will begin to rise, potentially erasing any savings that happened at the top of the funnel.
So, make sure that you are looking for reasonably well-targeted ads as you research your options; higher CPMs paid for more relevant traffic often yield big cost and time savings down the road.
Cost per Visit:
As with the above, relatively simple to measure: This is your total spend over the number of visits to your landing page.
Cost per Lead:
This is your total spend over the number of leads generated.
Cost per Pre-screen:
This is your total spend over the number of completed pre-screeners.
Make sure you are capturing the cost of administering pre-screeners at this step. While digital pre-screens cost your team nothing, many study teams still rely on having team members call or otherwise spend time speaking with leads as they go through the survey.
Capturing these human costs will help you understand where team bandwidth is allocated and reveal potential cost saving tactics (such as utilizing digital pre-screeners). These optimizations then immediately unlock your team’s capacity to actually run the research, rather than spending their time on responding to emails and talking with participants on the phone.
Cost per Passed:
This is your total spend (inclusive of team time and effort in pre-screening) over the number of people passing the pre-screener.
Cost per Consented:
This is your total spend (inclusive of team time and effort in pre-screening) over the number of people consenting to the trial.
Again, it is commonly the case that a team member spends time speaking with the potential participant to get their consent. This can also include the cost of a trial site, if the consent session has to happen in person. Ensure that you are capturing all costs in this measure.
Cost per screened:
This is your total spend over the number of people screened (inclusive of team time and effort in pre-screening) .
Some trials have a very light final screen process, in which a participant speaks to a medical director for a few minutes on the phone. Some have no final screening process. And still others require multiple visits and specialized equipment (e.g. MRI machines or other lab equipment) and the relevant specialists to run the procedure. All of these costs need to be captured for holistic analysis.
Earlier, we touched on the fact that the top of your funnel is essentially your warm welcome to interested parties, and that the bottom of your funnel is an exercise in hospitality.
When one looks the steps of the recruitment process, the clear line that divides the two phases lies at the Lead step. This is because leads are people who have read some of your message, understood your goals, and are ready to indicate a clear signal of interest in enrolling into your study.
What this means is that in the Welcome phase of your recruitment, your team’s goal should be to make your message easy-to-understand and quickly engaging; in the Hospitality phase, your team should emphasize speedy response, unambiguous in follow up, and clarity in next steps.
A benefit in folding the recruitment funnel into these two larger ideas is that it allows for two simple measures to quickly diagnose where leaks may exist in a recruitment bucket, and informs us on how to patch up the leaks.
The two measures we look to for quick diagnosis are heat maps and response times.
A heat map is a visualization of user behavior on a given landing page; it lights up heavily trafficked parts of a page in red, and leaves the unvisited portions blue. This design allows for a thorough understanding of aggregate visitor behavior with just a glance.
At Clara, we’re able to generate heat maps of the way visitors - in aggregate and individually - interact with our landing pages. We rely on this data to understand which of our pages' messages resonate, which stay ignored, and even which buttons and modules visitors interact with most. In short, heat maps help us understand how effective our welcome is to our target audience.
Here's a real-world example of how a heat map can diagnose an issue on a landing page in just a few minutes: One day, I noticed that a button that pops open a module for contact information capture was blue on my heat map, whereas the rest of the page was orange and read (i.e. well-engaged). I knew immediately that there was a problem in the way I’m presenting that opportunity to my visitors. As a corrective action, I chose to embed the capture form right at the top of a landing page to ensure that it is seen and acted on. Doing so led to a significant - and immediate - increase in my visit-to-lead conversion rate.
It could well be the case, however, that your team does not have access to a heat map, or that your IT team or vendor is unable to generate them for you on a regular enough basis. The good news is that there are now plenty of free tools on the market that helps with this work, and that there are a couple of common measures that you could utilize to replicate a heat map.
If you do not have ready access to heat maps, we recommend utilizing Google Analytics. Google Analytics is a free tool that is easy to embed into any landing page and features a dashboard that is fast, simple, and intuitive. Once you have GA tracking implemented into your page, you can use measures such as Average Time on Page, Average Number of Pages Per Session, Bounce Rate, and Exit Pages to uncover how visitors are interacting with your page.
For example, you might notice that your visit-to-lead conversion rate is lower than you’d like. If you see that a visitor only spends an average of 15 seconds on your page, it is likely that your content is overwhelming, or the layout of your page is confusing, and that the unengaged visitor is uninterested in finding out more.
The measure we use to diagnose how the Hospitality portion of a funnel is performing is response time, or the average time it takes a study team to get back to inquiries at each step.
More often than not, problems in getting cohorts from one step to the next in this part of the funnel are related to long response times. After all, the people finding a clinical study and attempting to apply are the most motivated sort; they simply will not wait 72 hours for a response without also investigating other studies, treatment options, and the like.
As a result, we hold ourselves internally to a 20 second response time. This means that someone leaving their contact information, or confirming that they’d like to schedule a consent session, or following up with a question about what the screening process looks like is responded to within 20 seconds. We are able to hold ourselves to this aggressive goal because we are able to automate quite a lot of this immediate outreach through our Clara platform.
However, most studies don’t have this capacity. Our guidance in this case is that your team responds to inquiries within 24 hours. It is not uncommon to see response times of 72 hours - sometimes, these lags can stretch for a full week or more - after the fact. Once again, a lead is motivated to find answers and options, and will use this ample time to research new treatments, find new research to get excited about, or simply lose interest.
In our experience, quicker response times have nearly always lead to significant lifts in conversion rates in the Hospitality phase of a team’s recruitment funnel. Optimize around this metric, and any team should see lifts in conversion rates.
The above framework places recruitment processes next to companion measures that can help identify and solve the most common problems in patient recruitment. Even if your analysis happens weekly rather than daily, ensuring that your team’s analysis is frequent and holistic can only yield benefits down the line. This means that you will find yourself recruiting more patients for less spend and effort, which frees up spend and effort for the important work of research.