Knowledge Translation From Disability Studies to Policy Makers: Literature Review and Expert Consultation

Mary Ann McColl, PhD, MTS

Canadian Disability Policy Alliance; Queen’s University


 


Aryeh Gitterman, EdD

Ryerson University

Dan Goldowitz, EdD

University of British Columbia

Abstract: Disability policy is currently receiving more attention than it has in the past 35 years. How have disability studies researchers participated in these processes, providing the results of their research to ensure the best possible evidence-based policy? This paper reviews the literature on barriers to knowledge translation from disability studies researchers to policy-makers, as well as the incorporating the recommendations of a high-level expert panel of experienced policy makers in disability portfolios. The principal barriers identified are: awareness of the policy process, awareness of government’s agenda, timing of information, format of the message, and commitment to the relationship. The panel offers five recommendations to address these barriers.

Acknowledgements

Our thanks to the members of the Expert Panel: Alex Bezzina (former Deputy Minister, Children and Youth Services and Citizenship & Immigration, Ontario); Bruce Rodriguez (former Deputy Minister of Education, Ontario, and CEO, Ontario Education Quality and Accountability Office); Aryeh Gitterman, former Assistant Deputy Minister in Education and in Ministry of Children & Youth; and, [one other who chose not to be named]. The Panel was supported by CIHR (SCA-145104) and CHILD-BRIGHT Network funding partners.


 


Knowledge Translation From Disability Studies to Policy Makers: Literature Review and Expert Consultation


 


Introduction

Disability policy is currently receiving more attention than it has in the past 35 years – since the early 1980s, when disability advocates triumphed in ensuring that people with disabilities were identified in Section 15 of the Canadian Charter of Rights and Freedoms as one of the designated groups whose rights were explicitly protected. Since that time, successive federal governments have instituted financial and tax measures to attempt to alleviate the economic burden on people with disabilities, but the elusive federal disability legislation that had been promised for decades did not materialize (McColl, 2019). Not until 2015 did legislative action begin to occur. Recently, the Accessible Canada Act was passed, and the federal government is currently addressing the challenges associated with implementation. Three provinces have also passed legislation aimed at removing barriers and promoting inclusion (Ontario, 2005; Manitoba, 2015; and Nova Scotia, 2017). Several others are in various stages of moving in that direction.

How have researchers participated in these processes, providing the results of their research to ensure the best possible evidence-based policy? How effective have they been at translating emerging knowledge about disability to those making decisions in government? How have they reached out to policy makers, including politicians, advisors, members of the public service, and administrative staff?

The emerging science of knowledge translation is aimed at communicating the best evidence from research to those who can use it to make a difference in the world. This process is significantly challenged when those transmitting and those receiving knowledge come from different worlds, as is often the case with researchers and policy makers (Gitterman, 2016). Although united in their desire to improve the life conditions and prospects of the people they work with and for, researchers and policy makers go about their business in different ways.

This paper is a reflection on the literature on this topic, and on the experience offered by a panel of high-level policy makers involved with disability portfolios in their respective Ministries. In December 2018, the panel convened to exchange ideas and tactics about effective knowledge translation in disability policy. The specific aims of the paper are:

a) To explore differences and similarities between the work of researchers and policy makers as regards people with disabilities;
b) To discuss the current literature on barriers to effective knowledge translation for evidence-based disability policy; and
c) To distill recommendations for researchers from experts involved in a recent consultation on policy relating to children with disabilities.

Similarities and Differences between Researchers and Policy Makers

Let’s begin by specifying who we mean when we refer to policy-makers. Policy makers are made up primarily of two groups – those who are elected to their posts and those who are employed in the public service. Elected officials/members of parliament are the decision-makers in a democratic government, since they are the representatives of the people. They are typically in their positions for four-year terms, and are dependent upon being re-elected in order to keep their jobs. This makes them particularly sensitive to the wishes and desires of their constituents. They are partisan in that they have a declared allegiance to a political party – either government or opposition. They may be aligned with a particular portfolio or ministry, or they may be members at large. Their expertise is in politics and government, and not necessarily in the substantive field of their portfolio.

Non-elected officials on the other hand include members of the civil service or government bureaucracy. These individuals must be non-partisan, since they continue in their posts even when the government changes. They typically have expertise, depth and history with the portfolio they work in. They conduct the day to day work of providing information to the politicians to support the work of government.

Similarities

The most important similarity between disability researchers and policy makers is their overall goal of making the world a better place, specifically for people with disabilities. People with disabilities are individual citizens, but they are often represented in the political arena by community groups, citizen coalitions or advocacy lobbies. Disabled consumers represent an authentic and essential voice in both research and policy. In research, consumer partners ensure that the evidence produced doesn’t miss the point, but instead confronts the real problems experienced in all their complexity. In policy, people with disabilities represent a sector of the electorate, with needs, preferences, and logistical considerations that must be addressed if disability policy is to achieve its desired effect.

Both disability researchers and policy makers address real-world social problems, and try to solve those problems by understanding them better. Both use evidence to inform their recommendations. Both have particular procedures that they typically use to ensure the quality and integrity of the evidence they consider and the decisions they make. Finally both have a primary duty to communicate what they have done, and what it means.

Differences

There are, however, important differences between disability researchers and policy makers that can result in miscommunications and misunderstandings in their attempts to work together. First, and perhaps most important, the accountabilitiesof researchers and policy makers are profoundly different. Researchers are accountable primarily to universities and funding agencies, as well as their own and their peers’ ideals of scientific integrity. Policy makers on the other hand, are accountable to their government and to the public. Whether elected, appointed or employed in government, policy makers in a democracy are responsible to the entire electorate through a duly elected government. This means that decisions must be made in such a way as to anticipate the approval of the people. It is worth noting that neither researchers nor policy makers are directly accountable to people with disabilities – that is, those who experience the problems that both seek to alleviate; however, since people with disabilities form a part of the electorate, policy makers are perhaps indirectly accountable to them and their support systems.

One of the inevitable tasks of government is to be re-elected, so as to be able to continue their work. Re-election in turn is dependent upon remaining in favour with the electorate. This reality creates a culture in government that is entirely unique, and foreign to any other sector of society. Policy makers must understand the needs and desires of a broad range of stakeholders, many of whom have directly conflicting positions. Furthermore, they must balance the wishes of powerful voices in society against principles like equity, justice, and security. In the case of disability policy, although recent statistics from the Canadian Survey on Disability tell us that people with disabilities make up 17% of the population, they seldom act in a block, as an influential political force.

A second difference between researchers and policy makers is the process they undertake in order to make decisions. For researchers, decision-making is meant to be independent of external forces, and held to the highest standards of methodological and theoretical objectivity. Every attempt is made to exclude outside forces that might contaminate the rigour or precision of the work.

Policy makers, on the other hand, must employ every means at their disposal to hear from as many sectors as possible in making decisions. Accountability to the public necessitates hearing from the public, and public consultations are an essential part in every policy making process. In disability policy, this means not only hearing from people with a broad array of disabilities, but also from families and support systems, service providers, equipment suppliers, health care workers, paid caregivers, other experts, and advocates.

Standards of evidence also differ between researchers and policy makers. What constitutes evidence for policy makers includes everything from individual letters expressing an opinion, to newspaper articles, to twitter posts, to committee deliberations, to scientific papers. Policy makers must consider all of these sources of information, and fold them into a coherent recommendation for action (or inaction) to solve important problems in society. All of these sources must be seen as representing the will of the electorate, and policy makers must evolve criteria for weighing the evidence they receive from stakeholders.

By contrast, evidence for researchers means something very specific. It means information that has been generated according to a rigorous, recognized and transparent process. It also means information that comes from a source with acknowledged authority by virtue of qualifications and track record. Finally, it means information appearing in a format and venue that is accepted by the scientific community, such as peer-reviewed journals and conference proceedings. Unless the information can stand up to the highest standards in all of these areas, it is discounted as not sufficiently credible, and dismissed.

Imagine if you can that someone formerly unknown to you sends you a piece of information about your research population – information that is only tangentially related to the research you are currently doing. This stranger insists upon the importance of this information, and further insists that you should drop the research you are doing, and switch your focus to address this problem. Perhaps they even infer that you don’t really understand the population you have been working with for many years, or that you don’t sufficiently care about them. What might your reaction be? This is the approach researchers take with government all too often.

This section of the paper relies on a review of literature on knowledge transfer between researchers and policy makers, with particular applications to the context of disability policy. Disability policy refers not only to policy that explicitly mentions people with disabilities, but also to those policies that apply to the general population, but have implications for people with disabilities. Disability policy is a particularly challenging sector for researchers, policy makers and consumers/advocates alike. Disability policy is spread across levels of government, across ministries within government, and across departments within ministries – making it challenging to coordinate and harmonize policy (McColl & Jongbloed, 2006).

Disability itself is a highly heterogeneous issue, including disabilities associated with sensory, cognitive, physical and psychological impairments. Disabilities may vary from mild (exerting only minimal effects on the person’s life) to severe (profoundly affecting every aspect of how a person engages with his or her world). Often, a policy that makes life easier for those with one type of disability doesn’t work for others or is even counter-productive for others.

Awareness of Policy Process

The most significant barrier that appears in the literature on knowledge translation to policy is a lack of awareness among researchers about the nature and process of policy work (Evison, Roy & Scarth, 2018). The policy environment operates on a different time frame, a different culture, a different reward system, and different resource constraints, from just about every other sector in society. Researchers have been characterized by policy makers as naïve in their understanding about the political agenda and the way government sets priorities, and arrogant in their assumed superiority of information, integrity, commitment (Oliver, Lorenc & Innvaer, 2014; Kothari, MacLean & Edwards, 2009; Dobbins et al., 2009; Ellen, 2018; Hyder et al., 2011).

A number of authors have shown that researchers carry damaging misconceptions about policy-makers – for example, that policy makers don’t care about the need for evidence (El-Jardali, Lavis, Jamal, Ataya & Dimassi, 2014); that they don’t have access to research information or databases (Ellen, 2018); that they are unsophisticated consumers of information (Gollust, Seymour, Pany, Goss, Meisel & Grande, 2017). Some researchers feel that government is unduly influenced by powerful voices, such as the medical lobby or the private sector (Ellen, 2018). Others mistakenly assume that policy makers only care about re-election and saving money (Gollust et al., 2017).

Not surprisingly, these assumptions leave researchers feeling alienated, frustrated and ignored. They become cynical about the policy process, and indulge negative ideas about the effectiveness of government. Undoubtedly, there are some governments that are overtly anti-intellectual, but for the most part, the research shows that 90% of policy-makers recognize the value and quality of research (El-Jardali et al., 2014).

The most troublesome feature of these mistaken assumptions is that they stand in the way of productive and constructive relationships between researchers and policy makers. The literature is unequivocal that the most effective insurance of transferring knowledge to policy-makers is a relationship with said policy makers (Jessani, Siddique, Babcock, Rothwell-Daey, Ho & Holtgrave, 2018), particularly one that is characterized by trust (Maddalena, 2015). The trouble is that researchers and policy makers don’t typically encounter one another in venues where they might cultivate these trusting relationships (Ellen & Brown, 2016). They don’t attend the same conferences, eat in the same cafeterias, walk the same hallways. In order to develop a relationship, they need to be deliberate and committed to seeking one another out (Boydell et al., 2017). They need to acknowledge their differences (Wall, McNie & Garfin, 2017), to focus on outcomes (Ellen, 2018), and to make time to meet face-to-face (Ellen & Brown, 2016).

Awareness of Government’s Agenda

A second significant barrier to translating research into evidence-based policy is a lack of awareness on the part of researchers about the government’s stated agenda and priorities. The main task of government is to identify the most pressing problems facing society at this time and place, to propose strategies to deal with those problems, and to deliver the kind of society we seek to live in. Those problems can range from border sovereignty, to national parks and nature preserves, to post-secondary education, to crime prevention, to housing for indigenous people, to stimulating employment, to creating opportunities for youth, to putting forward a winning Olympic hockey team. Regardless of how diverse these problems, it is the job of government to figure out which problems are most pressing, and to address those within budgetary realities. Researchers typically focus on a small sector of the population with whom they are closely aligned. Along with their consumer partners, they see the issues of this group in sharp focus, and often don’t understand how government can be insensitive to the very significant burdens borne by these individuals. They expect that by clearly enumerating the problems of this population, that government will see the error of its ways, abandon its existing agenda, and turn the ship around to focus on the needs of this group. When stated this way, it is probably evident that this is not going to happen. Instead researchers need to be aware of the priorities and directions government has promised to pursue and situate their issue within those parameters.

Timing of Information

A third barrier is the different time frames that researchers and policy-makers operate on. Researchers operate on relatively long timelines – one to several years to get a research grant, months to years to complete original research, more months to years to get findings published. In short, from inception to publication, producing evidence can easily take five years. Policy makers on the other hand operate on very tight (and often unpredictable) timelines. Governments seek to accomplish all that they promised in a typical four-year term, with the bookend years largely lost to transition to power and the necessity of re-election. Furthermore, the government’s legislative agenda has a fixed sequence into which information can contribute in a meaningful way. Policy-makers need the right information feathered into the process at the right point in the policy cycle – drafting, consultation, debate, review. Information that arrives out of sequence gets filed. It has no immediate relevance, because the issue addressed is not on the government’s current timetable, and so there is no logical home for it. What is the likelihood that a policy maker is going to be able to find just the right researcher with just the right information at just the right time to inform a policy decision? It is vanishingly small. Instead what makes sense is for a policy maker to have long-term relationships with researchers in whom he or she can trust, and who can be called upon when needed to supply information from his or her research archive that might be relevant to the problem at hand. It may not be the research being done right now that is most valuable and most influential in a current policy debate; but rather researcher’s immersion in a field that has suddenly caught the policy spotlight, and from which he or she can draw potentially relevant findings (Evison, Roy & Scarth, 2018, ; Oliver, Lorenc & Innvaer, 2014). 


Format of the Message

A fourth significant barrier to successful researcher – policy-maker interactions is knowing what information to present and how. Too often policy implications suggested by researchers amount to nothing more than “The government needs to do more for disabled people”, without addressing the specific parameters of what that more might look like. What policy makers most need is information that illuminates the consequences of choosing one policy option over others.

Numerous authors agree that the message needs to be simple and unambiguous (Maddalena, 2015; Tricco et al., 2016 ; Wickremasinghe, Kuruvilla, Mays & Avan, 2016; Marquez et al., 2018). Researchers have been acculturated to qualify everything they say with caveats, limitations and conditions (Oliver, Lorenc & Innvaer, 2014). But policy-makers need information that can assist in making a statement to the press or defending a particular policy choice to their Minister. They need credible results that can help to make the case for one policy option over another. Policy-makers don’t need a lot of detail about methodology or theoretical underpinnings. They need to know that the researcher can be depended upon, and that the research can help them make a case for one course of action over another.

Policy makers need statements of evidence that directly address the issue at hand in plain language – no acronyms, no jargon, no technical language; just concise, clear statements about the scope and magnitude of the problem, about implications of doing nothing in response to the problem, about demonstrated best practices in other jurisdictions, about consequences of different courses of action. This can be challenging for researchers, but it is essential. They don’t need a “sales pitch” or a persuasion campaign from researchers – there are advocates to serve that purpose. They don’t need a seminar or a lecture on the importance of an issue. They don’t need an economic analysis or business case – government has its own economic advisors to estimate financial viability and benefits. What they do need is information that will help them to weigh various policy options, decide what works, and substantiate that claim (Wall, McNie & Grifin, 2017). They are interested to be shown where inequities exist, the magnitude of the problem, and what those inequities look like in the lives of people. They are interested in local evidence from the jurisdiction over which they have authority, and any contextual information that can ensure the relevance of evidence.

Another common mistake that researchers make when talking to policy makers is to assume that the information they offer is objective and free of any political agenda (Gollust et al., 2017). Researchers need to own the fact that they too have an agenda, and their agenda is neither scientifically neutral, nor is it morally superior to that of government. Researchers seek to introduce information into the policy making process that will result in a favourable disposition of resources to the people about whom they care. Any time the allocation of public resources is at stake, the discussion is by definition “political”. Researchers need to acknowledge that information can be manipulated to favour a particular outcome, and to declare their possible conflicts of interest, thereby enhancing their credibility. Policy-makers are highly sensitive to incoming information that promotes a particular interest, and that information may be automatically discounted compared to information that is transparent and candid about its potential biases.

Commitment to Relationship

The final barrier to effective transmission of knowledge from researchers to policy-makers is time and effort – both of which are in short supply in both research and government. It takes dedicated time to cultivate and maintain a relationship with one or several key people in government (Jessani et al., 2017; Gollust et al., 2017). That time does not have direct metrics for productivity; consequently, few researchers bother to reach out to policy makers at all, never mind in a sustained and meaningful way over time. While 75% of policy-makers say they have a relationship with at least one researcher in their field, only 25% of researchers can say the same (Ellen, 2018 ; El-Jardali et al., 2014). 49% of researchers admit that they don’t specify policy actions that result from their work, and 88% don’t produce information tailored to the context of policy makers (Ellen, 2018).

Recommendations to Overcome Barriers to Knowledge Transfer in Disability Policy

How then do these two groups of people work together, as is so commonly required by research funding agencies and authorities? How do they create relationships that ensure the best possible outcomes for the people they serve? In December 2018, researchers and policy makers involved with childhood disability issues met to exchange ideas and tactics. The following recommendations for overcoming the above five barriers arose from that process.

Awareness of Policy Process

Evidence is a necessary but not sufficient condition for policy change - The panel encouraged researchers to understand the complexity of the policy environment and process, and to understand that evidence alone does not provide sufficient impetus for policy action. Factors like consistency with government’s agenda and priorities, compatibility with governing party’s values and ideology, public sentiment, media attention and many other factors (in addition to new evidence) contribute to motivating policy change.

Awareness of Government’s Agenda

Know the government’s agenda – Government’s agenda -- as spelled out in its election platform, Throne speech, Budget, position papers, media statements and website -- is a matter of public record. Researchers offering to help government to achieve its goals will have a much more receptive audience than those asking government to change its course.

Capitalize on momentum -- inertia is too difficult to overcome – Change that is already in motion is much easier to build upon than asking government to open a whole new area of activity and to focus on a group or an issue that they were formerly not active on.

Timing of Information

Be aware of budget cycle - If spending is involved, as it usually is, approval follows a very structured and time-sensitive process. To the extent that requests for spending conform to this cycle, they can be considered and planned for. If they arrive out of sequence, they are often lost by the time the next budget cycle comes around.

Focus on one-time spending opportunities - It is often easier to obtain one-time funding at the end of a budget cycle, than it is to obtain approval for an on-going budget line that represents a commitment over years to come. One-time funding can often be used for a demonstration project that builds commitment, evidence and momentum for subsequent action. The problem however is sustainability. One-time funding places a high demand on human and fiscal resources (especially in small organizations) to repeatedly renew or find new sources of funding.

Format of the Message

Don’t try to boil the ocean in a single pot - Many advocacy initiatives are defeated by trying to do too much in a single overture. It is very easy to keep adding aspects and elements to an advocacy proposal as everyone involved contributes their two-cents. It is important to keep the target small and obtainable, rather than grand and unlikely.

When one wins, we all win - Particularly in a constituency as varied and complex as people with disability, it can often happen that what serves one group does nothing for another, or even does a disservice to another. It is sometimes necessary to recognize that you can’t please all of the people all of the time. Instead, if meaningful change can be achieved for one group, it is important to recognize that as the world becomes incrementally more accessible and inclusive toward people with disabilities, we ultimately all win. This may require a commitment to focusing efforts on another group or a different need in the next overture.

Nothing succeeds like success - One way of reconciling situations where one subset of disabled people appears to have won while another lost, is to recognize that that win can be used to create an imperative to do something for the next group. A particular policy initiative that worked well and created positive reactions among the public can be an excellent springboard for the next overture.

Evidence is data in context - In order for research evidence to make sense in a policy environment, it must be situated in the context to which it is being applied. If the research wasn’t actually conducted in the jurisdiction, then it needs to be interpreted for the jurisdiction. Something that worked in Germany or Taiwan won’t necessarily work in the domestic jurisdiction. The contexts need to be compared to show how the evidence might pertain locally.

The policy environment is perfectly designed to produce the problems that people experience – This sounds somewhat sarcastic, but what it actually means is that when disabled people collectively experience a significant problem in their lives, it can inevitably be tracked back to a breach or inadequacy of policy. It is not enough to say that the policy environment is lacking; when seeking change, it is essential to be specific about where in the policy environment the problem lives.

Specific the exact policy change you want to see - Furthermore it is essential to be specific about the change you would like to see, based on the evidence. If you can state, for example, that the eligibility criteria for a certain program need to change from this to that, or that a certain regulated practice should be mandatory rather than optional, then the probability of change increases substantially.

Send the messenger hardest to ignore - The choice of messenger is a strategic one. While researchers may feel that they are the only ones who can accurately convey the intricacies and subtleties of their research, they may not be the most effective spokespersons for attracting the attention of policy-makers. Much more compelling is the messenger who has the intricate and subtle knowledge of the problem that is being addressed – specifically, the consumer. In terms of sheer credibility, consumers represent a much more effective messenger.

Say something - or else keep quiet - Represent your evidence with all the conviction and passion that motivates you to do the research you do. Figure out what is true based on your research and be prepared to stake your reputation on it. An argument that requires caveats, qualifiers, disclaimers, and limitations is not a very compelling argument.

Commitment to Relationship

Relationships take time - Panelists stated that the earlier they were involved in a research project or program, the better. If they had the opportunity to shape the question that was addressed, they were far more likely to be committed to the outcome or result. This did not mean that policy makers sought to influence methodology – that was acknowledged as the researcher’s expertise. It did, however, mean ensuring that the question was framed in such a way that the findings would be relevant to the issue confronting policy-makers.

Lose the hostility toward government - Our panelists expressed their surprise at the thinly-veiled hostility they sometimes encountered from people who were asking them for something – often money! The literature supports the dim view many researchers take of policy makers – of their motivation, their research literacy, their commitment, their understanding of the issues. It is not a great basis for collaboration, and researchers need to be honest with themselves about the implicit narrative they harbour about who policy makers are and what they do.

Remember… it’s a relationship - Finally, the panelists all expressed their commitment to valued relationships with researchers. They were grateful to have among their contacts researchers they could trust to provide them with credible, responsible information to illuminate policy discussions. Many of these relationships had crossed decades and had been sustained through transitions between jobs within government. These relationships were far more influential in a policy sense than any one-time information that crossed their desks.

Conclusion

This paper grew out of the deliberations of a panel of policy experts assembled to advise researchers and consumers/advocates involved with disability research. The advice that the panelists conveyed was remarkably consistent, and consistent also with the literature on knowledge translation to policy makers. The paper summarizes the similarities and differences between researcher and policy makers in the work that they do. It draws on the literature to list five key barriers to effective knowledge translation from researchers to policy makers. Finally, the paper synthesizes the panel’s suggestions for addressing each of these barriers. In summary, the panel concluded that researchers either need a crash course on government or need to work with consultants who can advise them on how government works. They need to become more familiar with the priorities and processes of the government they are attempting to influence. They need to learn to position information in such a way as to inform government processes, rather than simply presenting information in the way that they are accustomed to as researchers. And last but certainly not least, researchers need to recognize that information is most effectively shared in the context of a relationship characterized by trust and mutual respect. Such relationships take time to develop. They cannot be faked, and they cannot be summoned at a moment’s notice. Given that the incentives and rewards in the academic setting do not necessarily value the time spent engaging with policy makers, researchers who wish to influence policy need to do so for the somewhat altruistic goal of making the world a better place for the people they serve in their research.


 


References