# Presentation: 284389

## Getting FundedCornelia Beck

- Originally Presented
- March 7-9, 2001
- Houston, Texas

## Slide 2

## “I get by with a little help from my friends”

- I get “high” (ie funded)
- with a LOT of help from my friends.

## Issues in developing a research focus

- Estimating the research priority in your setting
- Articulating research goals
- Accepting rejection (the “Pink Slip Syndrome”)

## Why try to add research  to your already busy life? 

- To contribute to betterment of patient care
- To fulfill your creative instinct
- To have fun

## “Many people think that research is a supernatural gift of the Gods.It is simply an idea from a troubled mind, an inspiration followed by infinitely painstaking work and perspiration.”    Sir Frederick Banting    1891-1941

## Traits of a successful grant writer

- Good research skills
- Salesmanship skills
- Good communication skills
- Ingenuity and flexibility

- Administrative skills
- Interpersonal skills
- Persistence, dedication, patience, hard work
- Political awareness
- Integrity

## Tips for keeping focused	

- Practice saying “No”
- Remember that every opportunity does not have to be taken
- Don’t be taken in by:
- 	“But you’re the best one to do it”
- 	“It’s six months away”
- 	“I don’t know who else will do it”

## Slide 10

## Choose a specific focus
Get to know the experts
Be opportunistic
Schedule a set time each week
Discuss ideas with colleagues
Build synergistic dyads or groups

- Get to know the experts
- Be opportunistic
- Schedule a set time each week
- Discuss ideas with colleagues
- Build synergistic dyads or groups

- Start small
- Know when to go big
- Don’t expect to be the expert in every field, especially statistics
- Cherish your critics
- Target a specific date

## Getting started

- Engage Employer/Chair
- “Buy-in”
- “Release Time”
- Negotiate Resources
- Do
- Task List
- Time Plan

- Read one or more recent proposals in your field
- Have a funded proposal “at your side”
- Make a detailed outline before you begin to write
- Have two or three colleagues critique at multiple stages
- Simulate the review group

## Grant application process planning tool

- Preliminary development
- 		6-8 months prior to grant submission
- Proposal development and review
- 		3-4 months prior to grant submission
- (Crain, H.C. & Broome, M.E. (2000). Tool for planning the grant application process. _Nursing Outlook, 48_(6), 288-293.)

## Know your audience

- Who are reviewers?
- Your audience
- Busy people
- Agency’s
- Priorities
- Funding history

## Review criteria

- Significance
- Approach
- Innovation
- Investigator
- Environment

##      Be Sure You Care Enough Be Sure That A Funder Will Care Enough

## General principle

- Science2 + Marketing
- Art x pilot data
- Your end product _must_:
- 	be important
- 	be unique
- 	contribute significantly to the field!

- = Grantmanship

## What do you intend to do? Specific Aims

## Why is the work important?Background and significance

## What have you already done?Preliminary work

## How are you going to do this work?Research design and methods

## How will you know what you know?

- Data analysis

## Slide 26

## Specific Aims

## Sources of ideas

- Calls for proposals
- State of the Science
- Reports to Congress
- Integrated reviews
- Research reports
- What we found (did not ask/find)
- What it means (is there a “hole”)
- Implications for future research

- Anecdotal literature
- Case studies
- Clinical journals
- Clinical experiences
- Concerns
- Frustrations

## The biggest “Fatal Flaw”

- Bad Idea

## Good ideas

- Yours
- Passes the passion test
- You have capacity/resources to do it
- time, sites, institutional support
- Potential funder/s
- priority
- fit with portfolio/mission

## Important ideas

- Timely/Relevant
- New/Innovative/Unique/Creative
- Corroborative
- Contributive

## Doable ideas

- Realistic
- Affordable
- Resources required
- Time
- Money
- Availability of
- Sample
- Instruments
- Equipment
- Team/Consultants

## Ideas that sell

*Focused* sequence of studies that build on one another
- Studies that:
- are hypothesis-based
- provide useful info regardless of outcome
- info from one study sets stage for others
- have* contingency* plans if they don’t work
- Questions:
- What will be learned?
- Why is it important?

## There are no ways to make a bad idea look good, but there are a lot of ways to disguise a good idea 

## Purpose

- Aim for precision and simplicity
- What are you planning to do?
- Fit with aims of science (describe, explain, predict/control)
- “The purpose of this study is to.....” (what concepts? What relationships?)

## Purpose:

- The purpose of the proposed investigation is to test the efficacy of a family model of primary and secondary prevention for families identified as being at high risk for early coronary artery disease.  Primary prevention is defined as the reduction of the incidence of the disease while secondary prevention includes early detection and treatment of the disease.  Coronary artery disease is classified as early when it occurs in males before the age of 45 and females before the age of 55.

## Specific aims

- Briefly describe the problem your research will solve
- Clarify the long-term goals
- State the purpose of the study
- Clearly state the aims of the research
- Research questions/hypotheses
- Present the theoretical basis for your approach
- Briefly explain how you will go about it

- Provide a “preview”
- Start the “compelling story”

## Carefully specify outcomes

- Specify the effects that you expect to demonstrate. Do you want to:
- demonstrate effects at the mean?
- demonstrate effects at the individual level?
- show the numbers of person affected?
- show the chances of a person being affected?
- State how clinical significance will be determined

## Research questions/hypotheses

- The relationships among the specific constructs in your conceptual framework are the basis for the research questions or hypotheses

## Research questions/hypothesis

- Aim for relevance
- simple/complex: number of variables
- directional/nondirectional
- more/less - difference
- anticipated relationships
- statistical (null): no difference

## Early Experiments in Transportation

## Common weaknessesof specific aims

- The problem is not important enough
- Research will not produce any new or useful information
- Research is not testable
- Research aim is unclear
- Systematic constructs are not developed

## The description of the research and of its significance leaves the proposal nebulous and diffuse and without clear research aim
The hypotheses are ill-defined, lacking, faulty, diffuse, unwarranted, unimportant, or unimaginative

- The hypotheses are ill-defined, lacking, faulty, diffuse, unwarranted, unimportant, or unimaginative

## The problem fails to fall clearly within health-related research
The problem is scientifically premature and warrants only a pilot study
The research is overly involved with too many elements under simultaneous Investigation

- The problem is scientifically premature and warrants only a pilot study
- The research is overly involved with too many elements under simultaneous Investigation

## Hypotheses rest on insufficient, doubtful, or unsound evidence
The problem is more complex than the investigator appears to realize
The problem has only local significance
The problem is one of production or control

- The problem is more complex than the investigator appears to realize
- The problem has only local significance
- The problem is one of production or control

- It is unrealistic that the intervention proposed would ever be adopted in routine practice
- A descriptive study is proposed when the field is ready for an intervention study
- Cost issues have not been considered

## Slide 49

## Background & significance

## Background & significance

- Write this section first
- Convince readers that the research is important
- Provide a theoretical rationale for the research
- Clearly and explicitly state the hypotheses, variables and measures, and how the data analysis will confirm or disconfirm the hypotheses

## Background & significance

- What has already been done in the field?
- Why is this work important?
- What gaps will project fill?
- Why is this research important?

## Background & significance

- This is an important project
- That your agency should fund
- Background of present application
- Demonstrate knowledge of literature
- Critical evaluation of existing knowledge

## Background & significance

- Your chance to show that your work is important, how it is important, and how it fits into the larger picture.
- Evidence presented in this section should include a critical analysis of the literature.

## Framework

- Makes sense of concepts & relationships
- Conceptual framework
- Theoretical model

## Conceptual framework

- Clearly specify the conceptual domains
- Clearly identify the specific constructs in each domain
- Pay particular attention to the conceptualization and operational definitions
- Adequately identify the manifestations of constructs in your research

## To justify relationships among constructs use:

- Previous research
- Deductive logic of the conceptual framework
- Literature review

## Questions about conceptual issues

- Why did you choose these particular concepts and why are these concepts important as opposed to other concepts?
- What is the justification for specific hypothesized relationships among constructs in the proposed investigation?

## Evaluation ofconceptual framework

- How well you articulate & justify the conceptual framework
- How well you use this framework to formulate specific hypotheses, design the research study, choose measures, and specify a data analysis plan

## Presentation ofconceptual framework:

- Verbal Account
- Path Diagrams
- Venn Diagrams
- Regression Equations
- Structural Equation Models

## 	Social

- 	Factors

- Organizing
- Arrangements

- Physical
- Setting

- Technology

- Involve All Partners:

- Partnership
- Behaviors

- Organizational Performance

- Organizational Outcomes

- Quality Care

- Partnership Approach to Improving Quality of Care in Nursing Homes

- Organizational Environment

- Action

- Research

- Process

- Action

- Research

- Process

## Aspects of Care for Psychiatric Disorders

- Predictors of Use

- Use of Care

- Process of Care

- Outcomes of Care

- Patient
- Predisposing Factors
- Enabling Factors
- Need Factors

- Amount of Use
- No Use

- Appropriate
- Treatment
- Inappropriate
- Treatment
- No
- Treatment

- Service
- Substitution

- Disease-Specific
- Outcomes Symptom  Severity
- Generic Outcomes
- Functional Status

- Service System

- Costs of Care

- Modifying Factors

- ACCESS
- Availability
- Accessibility
- Accommodation
- Affordability
- Acceptability

- Direct Costs
- Indirect Costs

- Disease-Specific
- Prognostic Factors
- Generic Prognostic
- Factors

## Conceptual framework: Factors in the etiology of pressure sores

- Mobility

- Activity

-   SensoryPerception

- Extrinsic Factors

- Intrinsic Factors

- Tissue Tolerance

- Pressure

- Pressure SoreDevelopment

- Adapted from Bergstrom et al.4

## Literature review

- Research literature vs. narrative/anecdotal literature
- Convincing the reader
- Logical grouping of material
- Visible flow of material

## Literature review

- Prepare to discuss all critical literature
- Demonstrate a full command of the literature
- You cannot include everything, be selective
- Be sure to identify and include critical references
- Do be current
- Review must be organized
- When good reviews or summaries exist, use them

## CRISP - Computerized Retrieval of Information on Scientific Projects

- Extramural programs of:
- 	NIH (+ intramural)
- 	CDC
- 	FDA (+intramural)
- 	HRSA
- 	AHRQ
- http://www-common.cit.nih.gov/crisp/

## Give reader a “road map”

- “In this section, we review (1) the epidemiology of cocaine-associated presentations in the ED; (2) the pathophysiology of cocaine-associated chest pain; (3) other medical consequences of cocaine use; (4) the clinical treatment of cocaine-associated chest pain; (5) the clinical course of ED chest pain presentations with recent cocaine use; and (6) treatment-seeking and service by drug or cocaine users.”

- Saturation & Synthesis
- Concepts/relationships guide review
- What is: known & unknown
- Include
- Seminal articles
- Team’s work
- Reviewers’ research

- “A review of literature should be logically organized and, more importantly, should include a discussion of the major variables that are measured or manipulated in the research.  A ROL is a critical analysis and synthesis of related research.  It is not a simple listing of research findings.” (R-W, 1992)

## Organizing the Background and Significance

- Around the Specific Aims
- Around the major concepte

## Common weaknessesof conceptual framework

- Not clearly specified
- Too many constructs
- Overly simple
- No specific relationships among constructs

- Failure to consider alternative explanations
- Failure to discuss temporal relationships
- Literature not well integrated
- The conceptual framework does not direct any of the hypotheses

## I know you think you understand

- what you thought I said
- but what you fail to realize is
- that what I said
- is not what I meant

## Review criteria

- Significance
- Innovation

## Slide 75

## Preliminary studies

## Preliminary studies

- What has lead up to this project
- Why your team should get the money
- Your previous work
- Establish your credibility
- I have done...
- Our team has...

## Preliminary studies

- Your chance to show that you have the experience necessary to proceed competently on the proposed program of research
- You may discuss your own work--and that of your collaborators-- on reasonably related projects

## Preliminary studies

- Convince reviewers that you can achieve your research goals
- Provide enough detail of prior studies to convince reviewers of your technical competence
- Include any other information that will help establish your competence

## Pilot testing

- Single most important thing you can do
- A required investment
- Pre-test
- Instruments
- Procedures
- Case Acquisition
- Data Collection
- Analysis/Conclusion
- Answers the question: “Is this do-able?”

## Collect pilot datawhen you:

- Modify a standard instrument to increase its face validity or to get information more relevant to your research questions
- Modify procedures to use the instrument with a different population or in different circumstances than originally planned for the instrument

## Use an instrument on groups who either were not in the norming group on which the psychometric adequacy of the instrument was assessed or were under-represented in the norming group
Construct your own assessment instrument for one or more key constructs in your research

- Construct your own assessment instrument for one or more key constructs in your research

- Collect pilot data when you:

## Preliminary studies

- Don’t just list your findings
- Explain their importance
- What did you learn in the process?
- How did they prepare you to conduct current work?
- What pitfalls are you now aware of?

## Common weaknesses in preliminary work

- The investigator has inadequate experience or training for this research
- The investigator's previously-published work in this field does not inspire confidence
- The investigator relies too heavily on an insufficiently-experienced associate
- The investigator lacks skilled collaborators

## Remember:

- The quality of presentation of your pilot work is seen as a measure of your ability to present data from your proposed work

- .... the best predictor of what you will do tomorrow is what you did yesterday.....

## Review criteria

- Investigator
- Environment

## Slide 88

## Slide 89

## Schematic of Trial Design

**Schematic of Trial Design**

- Physical Therapyand Placebo

- Fluoxetine andFriendly Visit

- Physical Therapyand Fluoxetine

- Friendly Visitand Placebo

- Interventions continuedat maintenance intensity

- Usual Care

- UsualCare

- Usual Care

- 12

- 9

- Maintenance Phase

- 3

- MONTHS

- 3

- RandomizedControlled Trial

- 0

- Determination of Treatment Response

- Yes

- Yes

- Yes

- Yes

- No

- No

- No

- No

## Common weaknesses of  setting & sample

- Inadequate institutional setting
- Restricted access to patient populations
- Sampling technique not adequately specified
- Sample too heterogeneous to yield desired results
- Size inadequate, unrealistic or unattainable

## Common weaknesses of  setting & sample

- Inappropriate composition, number, or characteristics of sample
- Criteria for subject selection not justified
- Subject selection bias and attrition not addressed

## Common weaknesses of  setting & sample

- No assurance of ability to recruit adequate numbers of subjects
- No projected dropout rate based on previous experience
- Experimental and control groups not comparable

## Intervention

- 1.  Description of protocol
- 		Replicability
- 2.  Pilot testing
- 		Feasibility

## Intervention

- Tx  Fidelity/ Adherence
- Dose response
- Practicality
- Interventionist
- Resources needed

## Common weaknesses of intervention

- Not adequately described
- Not adequately supported by literature
- No theoretical underpinnings
- Not practical

## Slide 97

## Slide 98

## Is the conceptual link between the intervention and the outcome variable logical?
To what extent is the outcome variable amenable to change?
Is the content validity of the outcome measures adequate for detecting the effect of the proposed intervention?
Does the instrument used to measure the attribute have a potential distribution of scores that will allow detection of change?
Stewart, B. & Archbold, P. 1992

- To what extent is the outcome variable amenable to change?
- Is the content validity of the outcome measures adequate for detecting the effect of the proposed intervention?
- Does the instrument used to measure the attribute have a potential distribution of scores that will allow detection of change?
- Stewart, B. & Archbold, P. 1992

## Beware of threats to validity

- Threats to internal validity
- Selection
- History
- Maturation
- Mortality
- Testing
- Diffusion of Tx
- Regression to the  mean

## Threats to the statistical conclusion

- Low power
- Violated assumptions
- “Fishing”
- Reliability to measures
- Reliability of Tx

## Threats to external validy

- Interaction between Tx and selection
- Interaction between setting and Tx
- Hawthorne effect
- Experimenter effect

## Common weaknesses of methods

- Design flaws
- missing control groups
- control groups are the wrong ones
- Insufficient statistical power
- Imprecise measures (dependent variables)
- Cannot recruit the needed population
- Lack of face validity

## Common weaknesses of methods

- Studies lack cohesiveness
- Sequence of experiments is not logical
- Results of a study leads to a dead end
- Subsequent studies rely too much on previous experiment
- Contingency plans either not stated or ill conceived and are not feasible

## Common weaknesses of variables

- Confounding variables not controlled sufficiently
- Controls are either inadequately conceived or inadequately described
- Tool development insufficiently conceptualized; underestimation of difficulty

## Common weaknessesin measurement plan

- Measures not linked to constructs in conceptual framework
- Key constructs inadequately measured
- Tools not clearly specified; too ambiguous to obtain data needed

## Common weaknessesin measurement plan

- Tools inappropriate for variables as developed
- Need multiple measures of some constructs
- No mention of test-retest gains

## Slide 108

## Seriously consider respondent burden

- Respondent burden will differ in accord with the respondent’s status and other obligations
- In many studies, research subjects should be paid; marginal costs may be small
- What will the research subject need to be told and who is going to do the telling

## Table 1: Proposed Measures

## Slide 111

## Slide 112

## Common weaknesses of procedures

- Lack of attention to logistical realities of conducting study
- Role of PI as data collector raises concerns about bias

## Slide 114

## Data analysis

## Data analysis

- Answers research questions
- Results will achieve aims
- Descriptive statistics first
- “Look before you regress”
- Hypothesis testing
- Other associations

## Common weaknesses of data analysis plan

- The statistical aspects of the approach have not gotten enough consideration
- The data analysis plan does not describe how specific hypotheses will be tested or analyzed
- The data analysis plan lacks congruence between data analysis procedures and aims of the study

## The Investigator does not understand the proposal's complex statistical procedures
The Investigator uses simple statistical procedures when more complex procedures are necessary or commonly used

- The Investigator uses simple statistical procedures when more complex procedures are necessary or commonly used

## The Investigator uses statistical procedures that are not state of the art
Analysis inappropriate for the design of data collection methods
Approach for handling missing data excluded

- Analysis inappropriate for the design of data collection methods
- Approach for handling missing data excluded

- Failure to provide psychometric data for measures
- Failure to consider psychometric adequacy of measures
- Uncontrolled measurement biases

## Don’t forget:

- Limitations
- Dissemination plans

## Allow time to review your proposal

- Develop a proposal map schema to detect any gaps (e.g. unrelated parts) in your proposal
- Using your proposal map, have a colleague attempt to map your grant proposal

## “Mapping” your proposal

- Concepts
- Background
- Data
- Analysis
- Interpretation
- Budget

## Slide 124

## The title is your work in a nutshell

- Descriptive
- Specific
- Appropriate

## The abstract 

- May be the only part read by many “voters”
- The broad, long-term objectives of the proposal are....
- The specific aims are: 1)   , 2)   , 3)
- The significance of this project is...
- The research design is........

## Putting it together

- Remind reviewers of priorities & how your proposal fits with priorities
- Indicate how you can save $$
- Use
- 	readable
- 	font

## Remember to:

- Use grammar/spell check
- include reliability of  variables & power calculations
- Sell your idea
- Use headers
- Have someone else review

- Follow rules
- Page limitations
- due by date
- Signatures
- Contents

## Slide 130

## Most common reasons for disapproval

- Lack of new or original ideas
- Diffuse, superficial, or unfocused research plan
- Lack of knowledge of published relevant work
- Lack of experience in the essential methodology
- Uncertainty concerning the future directions

- Questionable reasoning in experimental approach
- Absence of an acceptable scientific rationale
- Unrealistically large amount of work
- Lack of sufficient experimental detail
- Uncritical approach

## The Investigator is spreading herself too thin; she will be more productive if she concentrates on fewer projects
The Investigator needs to communicate more with colleagues 
The past connections between the various investigators remain unclear

- The Investigator needs to communicate more with colleagues
- The past connections between the various investigators remain unclear

## Characteristics of fundable proposals

- A good research idea
- A well-focused, well-written research proposal
- A good “Track Record” and/or substantive preliminary results
- A substantial percent effort
- A stable work group
- An ample number of substantial, preferably peer-reviewed publications

## Take home messages

- Plan, Plan, Plan
- Allow enough time
- Importance of abstract

## “Many people think that research is a supernatural gift of the Gods.It is simply an idea from a troubled mind, an inspiration followed by infinitely painstaking work and perspiration.”    Sir Frederick Banting    1891-1941

## “I get by with a little help from my friends”

- I get “high” (ie funded)  with a LOT of help from my friends.

## General principle

- Science2 + Marketing
- Art x pilot data
- Your end product _must_:
- 	be important
- 	be unique
- 	contribute significantly to the field!

- = Grantmanship

## Some advice

- Be:
* *Proactive
-  Persistent
-  Patient

## Be careful what you ask for - You just might get it

## Slide 141

## Good Luck!

## Assemble a Research Team

- Project management plan
- Roles/responsibilities of team members

## Who do you need on your team?

## Principal investigator

- Has original idea that stimulated project
- Has major responsibility for scientific integrity and financial status
- Typically has 25-40% on project

## Co-principal investigator

- Integral to development of project
- Takes a leadership role for implementing portion of project
- Typically 10-25% on project

## Co-investigator

- Has responsibility for smaller portion of project
- Typically 10-25% on project

## Consultant

- Has specialized area of expertise
- Up to 5% on project
- Limited or concentrated effort

## Project staff

- Work in specific roles
- 25-100% effort on project