Research is Risky Business: Effective Risk Assessments for Your Research Compliance Program

September 3, 2024

by Sarah M. Couture and Katherine Cohen

Risk assessment may be somewhat of a buzzword in compliance circles. Compliance officers know their compliance programs should be based on risk, but some may not know where to begin when it comes to developing an approach to risk assessment. The charge is no different for research compliance programs; some risks are very distinct. Clinical research is a risky business, and effective research compliance programs must be based on prioritized risk. This article will explain what a risk assessment is,why it is essential, how to use risk assessment results in the research compliance program, and approaches and considerations in performing a research compliance risk assessment.

What is a risk assessment, and why is it essential?

The U.S. Department of Health and Human Services (HHS) Office of Inspector General (OIG) defines a risk assessment as “a process for identifying, analyzing, and responding to risk” and as a process that “looks at risk to the organization stemming from violations of law, regulations, or other legal requirements.”^[1] Every compliance program will approach their risk assessment differently. The key is to have a defined process that looks at a variety of sources to identify potential risks and uses a consistent methodology to evaluate and prioritize those risks. No compliance team can address every risk for their organization. The risk assessment process allows the organization to ensure they are aware of their risks and have decided how to allocate their resources to address the highest priority risks.

The risk assessment process is a critical component of an effective compliance program. The updated OIG General Compliance Program Guidance lists it as one of the seven elements of an effective compliance program (Element 6 – Risk Assessment, Auditing and Monitoring). Because of this, being able to show your risk assessment process and risk mitigation activities can help decrease or prevent potential consequences in case of enforcement action. It is also a vital tool to help your compliance function and the organization be proactive rather than reactive. Risk assessment is an expected activity for an effective compliance program, but it can also be used to measure effectiveness of your program. The process can help you identify and prioritize risk to best allocate resources towards the highest priority risks. Your risk assessment activities can also help you identify and reduce fraud, waste, abuse, or other compliance risks. One of the most important reasons to prioritize risk assessment is that it can help you engage the operational teams in their compliance responsibilities, foster collaboration between compliance and operations, and form or improve relationships. It is also a vital tool that establishes accountability for those who govern the organization, as the document should be reviewed and approved by your compliance committee and governing board(s).

Performing a research-specific risk assessment in the clinical research setting is also imperative. While all the reasons to complete a risk assessment already listed here also apply to why you should complete a research-specific risk assessment, there are additional reasons to engage in this exercise for your clinical research program. First, research adds an extra layer of rules and regulations that must be complied with and that creates risk for your organization. The risk assessment helps you identify which risks you have based on the specifics of your research program and categorize their priority within the context of your other organizational risks. Engaging in the research risk assessment exercise also communicates to your organization, partners, research sponsors, and research participants that the organization values and takes steps to protect the integrity of research and the individuals who participate in research projects. Ensuring the organization performs research risk assessment is also a way to proactively ensure the appropriate use of federal funding you have received for research and protect your organization’s ability to continue to receive federal funding for those purposes.

What to do with the risk assessment

It is clear that risk assessment is essential to ensure the research compliance program is effective, but once the process is launched, what should the program do with the risk assessment? First—and certainly the goal related to risk assessment that is most commonly discussed—use it to guide your annual research compliance work plan development. The risks that are the highest priority as assessed by the risk assessment are those that the research compliance program should focus on. What kind of audit or evaluation will help you gauge compliance in that risk area? How can you audit to ensure the risk is appropriately controlled? Are sufficient controls like policies, procedures, education and training, and monitoring in place to help manage the risk? How can you “test” the risk area to see if the organization functions compliantly regarding the risk? The work plan outlines what your program will prioritize in the upcoming year. What is significant enough to prioritize while dealing with your day-to-day role? And how many of the highest priority items from the risk assessment should the compliance program include in the work plan? These are questions of resources.

Only include tasks, audits, or evaluations in the work plan that you have staff time and resources to complete. Organizations with small compliance teams may have a small number of risks. It is better to start with fewer work plan items and be able to finish them than to include more than your program can handle and have to explain why the work plan items were not completed. Second, use it to inform your education and training approaches. Educate your board, leaders, and compliance committee on the universe and prioritization of the risks. This is your organization’s research compliance risk profile, and leaders should understand it. Also, use it to evaluate your research compliance education and training plan each year. The risk profile should drive the education and training approach and content, so ensure both annual general research compliance training and risk-based training sessions are updated according to the risk assessment. Third, keep your research compliance program structure, as in how you have built and implemented the seven elements, aligned with the risk profile. At least annually, evaluate your program elements, such as the code of conduct, policies, procedures, etc., according to the risk assessment to ensure they are updated and aligned with the current risk profile. This could be part of your annual assessment of your program’s effectiveness. Finally, ensure the risk assessment itself stays updated. Risk assessment is a dynamic process that should remain updated as the regulatory landscape and our organizations change.

Universal research risks that should be considered in your research risk assessment

Each organization’s risks will vary based on the type of research conducted, the type of funding received, and the resources dedicated to the research operations. A comprehensive research compliance risk assessment should include a review of the primary risks most research organizations face. First, examine research integrity and HHS Office of Research Integrity (ORI) requirements for those that receive federal funding. For issues that trigger the research integrity regulations, review your policy, history of annual reporting, and education of staff. If you have received allegations of falsification, fabrication, or plagiarism, how have those been reviewed, and did your process align with the regulations? Conflict of interest (COI) regulations and rules may vary depending on the funding source of the research you conduct. Evaluate the risks created by multiple COI standards, where you evaluate the potential for COI in research, how you implement management plans, and whether you monitor compliance with those plans. Organizations that receive federal grants should evaluate the processes for complying with the COI requirements for subrecipients. Do not forget about institutional COI (ICOI) when looking at COI risks. If you do not have a separate ICOI policy, how might an unidentified or unmanaged ICOI risk impact your organization? If your organization receives federal research grants, how are the grant and financial compliance risks managed, such as compliance with the uniform guidance and effort reporting obligations?

In human subject research studies, what types of regulations is your organization subject to? Is it only the Common Rule (HHS Office of Human Research Protections (OHRP)), or is it also U.S. Food and Drug Administration (FDA) regulations? Are you conducting other human subjects research with different regulatory requirements? How would your organization address complaints of a research participant? Are those studies required to comply with good clinical practice rules and, if so, how are you monitoring compliance with those requirements? Have you evaluated the privacy risks posed by your research program? Is HIPAA-covered data included in research being conducted, and if so, how? Do other privacy regulations, such as the General Data Protection Regulation or the California Consumer Privacy Act, apply to your research? If there was a privacy incident in the context of a research study, who handles the evaluation of those incidents?

How do you manage the risk involved in that billing process for clinical research studies involving services that can be billed to Medicare or other insurance providers? Are you appropriately identifying the research participants to ensure the billing of services provided is compliant in the context of a research study? Are you completing a coverage analysis and ensuring it aligns with what services you are billing? Do you have a process for putting the proper codes, diagnosis codes, and modifiers on the claims for the research services? Clinical research billing processes often depend on siloed functions working together. How is this process managed to ensure communication across those silos?

For organizations conducting research with animal subjects, does your Institutional Animal Care and Use Committee comply with all applicable requirements? Do you know which animals you have in your facility and the different regulations that might apply to you based on the types of animals you are working with?

Research organizations face risks like biosafety, export control, and data security. This is not intended to be a comprehensive list of risks for research organizations but rather a place to start for those who might be asking themselves where to look for risks within the research program.

Risk assessment considerations: Building your process

The risks discussed above are related to specific research regulations, and each of these risks should have appropriate controls in place. Insufficient controls are a risk in themselves. In addition to the research regulation and related controls risks, there are both external risk factors and internal risk factors that must be overlayed and included in your research compliance risk assessments. External risk factors to consider include recent enforcement actions related to research risk areas, advisory opinions, guidance documents, government reports or updates, OIG Work Plan items, or priorities of other relevant research-related government agencies. Internal factors are those that are specific to the organization. These may include culture-related issues, such as transparency, accountability, and fears of retaliation; specific issues in certain departments; turnover rate of staff in risk areas; results of recent audits—whether conducted internally or externally; an organization’s specific software systems used; and the physical plant and facility, including consideration of how buildings, equipment, security, etc., impact risks. There may be other internal factors to consider at your organization. With some thought, it becomes easy to see how these external and internal factors could impact your risk profiles and why they should be part of your risk assessment considerations.

As the introduction discusses, there is no one way to conduct a compliance risk assessment. The approach should be one that works for the organization and incorporates several concepts: subjective input, objective input, external factors, internal factors, risk tolerance, prioritization, diverse perspectives and collaboration, ongoing evolution, and documentation.

Subjective input: Ensure your approach includes a way to gather subjective input from stakeholders. Stakeholders include those with a perspective on the research compliance risks in your organization; they should consist of leaders, managers, and investigators from the various risk areas in your research operations. These may include but should not be limited to research finance, research coding and billing, grants management, research administration, legal, HR, marketing, clinical areas conducting research, study coordinators, etc. Some organizations conduct interviews of stakeholders to gather their perspectives on risk, while other organizations choose to send a risk assessment survey asking about perspectives on research compliance risks the organization is facing. Interviews allow the opportunity for discussion, while a survey allows for collecting a wider group of perspectives. A combination of these activities may be beneficial.
Objective input: While subjective input is essential as it accounts for the risks that operational and finance stakeholders may see or know about, objective input is necessary to gather risks seen in data or other documents. Perform a document review to catalog risks. These documents may include but should not be limited to internal or external audits results, risk-related data, policies, documentation of controls around risks, etc.
External factors: Include in your risk gathering risks seen in recent enforcement actions, guidance, government agency bulletins, recent regulation changes, etc.
Internal factors: Include in your risk gathering any internal factors or influences that may affect risks or are risks themselves. These may include but are not limited to, culture or morale issues in a specific location or department, turnover rates, personnel specifics, software or other systems, or other organizational specifics that may impact risk.
Risk tolerance: Before an organization can prioritize risks, it must understand and even document its risk tolerance. A specific financial penalty amount may not significantly impact a larger organization but could be traumatizing for a smaller one. Discuss with your leadership their perspectives on reputational, financial, legal risks, etc., to ensure relevant leaders share views of what is considered low, medium, and high risk.
Prioritization: A risk assessment includes not only risk gathering and cataloging but also prioritization. Without prioritization, an organization cannot understand its high, medium, and low risks.
Diverse perspectives and collaboration: It is essential to get a variety of perspectives on the risks themselves and ensure a variety of perspectives on prioritization. Also, collaboration on the research compliance risk assessment must be ensured. Work with your research compliance committee if you have one; if not, recruit three to five research leaders to assist you with the process. This ensures a variety of perspectives are considered and helps promote engagement and buy-in with research operations.
Ongoing evolution: It is obvious when you think about it, but a risk profile is something that will evolve. As risks come and go, become more or less important compared to other risks, and as organizations and the regulatory landscape evolve, risks change, so risk assessment processes must build in evolution. How will risks be gathered between the regular (i.e., annual) risk assessment? How will they be tracked and documented? How will they be prioritized to weigh them compared to other risks? Many organizations include regular risk assessment discussions as part of their routine compliance committee activities. Again, this ensures the right perspectives are considered and further engages stakeholders.
Documentation: Ensure your process for risk assessment is documented so that it can be consistently followed. Also, ensure all aspects of performing your risk assessment are documented to capture and evidence the process.

Before you develop a stand-alone research compliance risk assessment, determine if other risk assessments are taking place in the organization, whether general compliance, enterprise risk management (ERM), or otherwise. It will be wise to learn from these approaches and potentially adopt one, tailor it for research risk, and/or collaborate on the risk assessment.

Performing the research compliance risk assessment

As you begin your research compliance risk assessment, start by gaining an understanding of your research portfolio. What kind of research are you doing? Based on the types of research at your institution, what regulations and rules apply to you? What is the maturity of your research program and the research administration/operations infrastructure? Also, engage your research compliance committee in the process. Ensure that they understand both the risk assessment process and their roles in the process. Get their input on your proposed risk assessment approach, including the risk gathering, cataloging, and prioritization process. During regular meetings, keep them informed throughout the process and discuss findings, observations, and recommended pivots. Leverage their perspectives in the draft research compliance work plan that results from the risk assessment.

As you begin to gather risks, start with research leadership. Set up time to discuss their perspectives on the research compliance risk assessment as well as their perspectives on the risk profile. What is top of mind for them? Where are the areas they think you have risk? What keeps them up at night? After gathering risks from leaders, talk to the research operations, finance, and regulatory “doers.” What gaps do they see? What concerns do they have? What do they feel unprepared to handle?

As you gather these subjective perspectives, begin to look for objective evidence of risks. Have there been audits with findings at your organization, such as FDA inspections or billing audits? What data in your research infrastructure can be reviewed to identify potential risks?

Then, evaluate eternal risks. Look at recent enforcement actions—what themes can you draw out? What parallels do you see to your organization? Review ORI, OIG, FDA, and OHRP enforcement and reports. And consider internal specifics that are risks or could impact risks.

As you are gathering the risks, you will be documenting, literally listing, them. Develop a compliance risk tracking tool that can be leveraged for ongoing risk assessment. This can be as simple as a spreadsheet tool developed internally or could be part of your compliance program management software tools. Many tools have been shared in HCCA presentations or on the HCCA.net social site. Explore these options and consider tailoring one that works for you. If this is your first time working on a risk assessment, you’ll likely be surprised at how many risks are mentioned by those you interview and/or survey and that are identified from internal document review and external activity review. Let both your compliance committee members and leaders know that it is very normal to have up to 100 or more risks identified in the risk assessment.

Once the risks are cataloged, it is time to rank them. Ranking the risks is how we get the prioritization. Use your existing ERM prioritization approach, or if one does not exist, develop a methodology to rank the likelihood of each risk happening, the impact each risk could have if uncontrolled, and the sufficiency of any existing mitigation/controls that may be in place around each risk. Many organizations use a 1–5 ranking system, with each number having defined criteria for the likelihood, the impact (1 being lowest likelihood/impact, and 5 being the highest), and the sufficiency of existing controls (1 being no controls, and 5 being sufficient controls). Compliance and risk professionals have shared many tools online that can serve as inspiration. Note that likelihood and impact scores would increase a ranking, while sufficiency of internal controls would decrease a ranking: risk ranking = (likelihood + impact) - mitigation. It is critical to have both compliance and operations and finance perspectives on the ranking, as this provides a balanced outcome. Leverage four to five members of your research compliance committee to separately rank the risks, then average the scores. Ranking the risks for the first time will feel challenging, but the process becomes more natural on subsequent assessments (i.e., in following years). Once the risks are ranked, put them in numerical order. This will display the rankings from highest to lowest and provide the organization with an understanding of its research compliance risk profile.

Then what?

As previously discussed, the risk assessment results will be used to develop the annual research compliance work plan. This work plan becomes a true picture of what the compliance program is doing to evaluate the organization’s highest risks. Present both the risk assessment results and the resulting work plan to the research organization’s governing body. As compliance tests or evaluates the risks and the controls in place, remember that research operations and/or finance develop and implement the corrective action plans to help mitigate the risks. These discussions and activities can be part of the work performed by the research compliance committee.

Keep the risk assessment updated with your research compliance committee. New risks will surface as older risks go from a higher ranking to a lower one as the result of operations’ mitigation activities. On a regular basis, update the overall risk assessment by again gathering perspectives, examining documents and data, and considering outside regulatory activity and internal factors. Because the tools developed for the initial risk assessment will be leveraged, subsequent risk assessments will likely feel less onerous. Keep leadership and the board informed of the changing risk profile and the work compliance that is being done to evaluate the risks and operations and finance mitigation activities.

Developing and performing a risk assessment for the first time is both a significant and rewarding job. The insights it brings to the compliance program, the engagement it fosters with operations and finance, and the organizational risk mitigation will prove that the risk assessment is well worth the effort.

Takeaways

Understanding the organization’s research risks is an essential foundation for the research compliance program.
Research risk assessment should be a collaborative process between research compliance and research operations leaders and managers.
The organization should ensure the risk assessment process includes approaches for keeping the risk assessment and risk profile updated as the organization’s risks change over time.
The prioritized research risks are used to develop the research compliance work plan.
The research compliance program should keep research leaders and the board of directors informed about the risk assessment process and the organization’s evolving risk profile.

1 U.S. Department of Health and Human Services, Office of Inspector General, General Compliance Program Guidance, November 2023, 55, https://oig.hhs.gov/documents/compliance-guidance/1135/HHS-OIG-GCPG-2023.pdf.