WayToServe Plus: In-service Professional Development Component to Improve Responsible Alcohol Service Ph II
Phase: NA
Conditions: Alcohol Drinking, Alcohol Intoxication, Alcohol Use, Unspecified
Sponsor: Klein Buendel, Inc.

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
    
Title:  
WayToServe Plus: In-
service Professional 
Development Component 
to Improve Responsible 
Alcohol Service 
 
 
Protocol #: 1R44AA029364 - 01  
 
 
Date: 
10/2022 
 
 
 
CLINICAL TRIALS COVER PAGE 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Cover Page 
   1667 Cole Blvd. Suite No. 220 
   Golden, Colorado 80401 
  Main Phone: 303.565.4321 
 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
WayToServe Plus: In-service Professional Development Component to Improve Responsible 
Alcohol Service 
1R44AA029364 - 01 
 
Responsible beverage service (RBS) training for alcohol servers is a promising intervention for reducing driving while 
intoxicated (DWI) by alcohol. Training, certification, and in-service contact improves professionalism and effectiveness 
of prevention interventions delivered by community members such as alcohol servers. This SBIR Fast-track project will 
develop and test an in-service professional development component to the WayToServe® online RBS training to 
improve the effectiveness of RBS training in order to make further gains in reducing problem alcohol behavior in 
communities.  
 
2. Specific Aims 
Driving While Intoxicated (DWI) is one of the most preventable public health risks in the United States. While progress 
has been made (21,113 DWI deaths in 1982 to 10,511 in 2018 [latest available data2]), DWI death rates remained 
relatively unchanged in 2016 and 2017.3 While new policy and interventions are needed to reduce social, legal, medical, 
and economic consequences of DWI, gains are possible by increasing efficacy of existing evidence-based interventions. 
Among existing strategies, responsible beverage service (RBS) training4-6 has been effective in some cases.7 We have 
shown that an online RBS training, WayToServe®, was effective in two randomized trials, funded by the National Institute 
on Alcohol Abuse and Alcoholism (NIAAA; R01AA14982, R01AA016606). Onsite (bars and restaurants) and off-site 
(package stores) premises receiving WayToServe had higher rates of refusing sales to pseudo-intoxicated patrons (PiP) 
than control premises11 (see Box 1). We are testing a culturally-tailored WayToServe to improve its efficacy with 
Spanish-speaking servers and clientele, with a grant from the National Institute on Minority Health and Health 
Disparities (R44MD010405).  
 
The goal of this Fast-track SBIR research is to develop and evaluate an in-service professional development 
component to increase the efficacy of our evidence-based WayToServe RBS training by improving alcohol servers’ 
professionalism and RBS skills. The rationale arises from literature showing that training and certification of community 
members improves professionalism12 and, along with in-service contact, has boosted success of community prevention 
interventions.13,14 Alcohol servers will benefit from in-service contact aimed at improving their professionalism by a) 
motivating them to implement RBS skills in the face of common barriers, such as pressure to sell, management 
disinterest or resistance, and customers’ attempts to continue being served, b) receiving support for RBS from a 
“community” of alcohol servers, especially for servers who work in small or unsupportive premises, and c) preventing 
natural degradation of skills over time. Scientific premise derives from our research establishing the effectiveness of 
WayToServe online RBS training11 (see Box 1) and preliminary survey exploring the in-service component concept (see 
Box 2).  
 
Phase I specific aims will establish the feasibility of the WayT0Serve Plus in-service component by: 
1. Developing content of in-service professional development on RBS skills with a) premises owners/ managers 
interviews and b) alcohol servers in California, New Mexico and Washington State trained by the WayToServe 
online RBS training through focus group discussions, and survey data.  
2. Producing a prototype WayToServe Plus in-service professional development component in flowcharts and an 
initial library of social media posts, including text, graphics, and interactive elements and evaluating it for 
acceptability, usability, and engagement in focus groups and field pilot trial with alcohol servers in California, New 
Mexico and Washington State trained by the WayToServe online RBS training. 
3. Determining feasibility of WayToServe Plus in-service professional development component and developing 
evaluation plans for Phase II research. Feasibility will be indicated by WayToServe Plus being acceptable/desirable 
to 75% of owners, managers, and alcohol servers; having specifications and prototype successfully developed; 
being usable (minimum 68 out of 100 on System Usability Scale67), engaging most servers (75% read and 50% react 
to a post), and improving theoretical mediators of RBS skills (d=0.2). 
Milestones will be: 1) EAB-approved Phase I research plans and development of the content outline, 2) development of 
WayToServe Plus prototype, 3) evaluation of the WayToServe Plus prototype, and 4) 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
feasibility assessment of the WayToServe Plus in-service professional development component.  
 
Phase II specific aims of the SBIR research will achieve the milestones of full production of WayToServe Plus in-
service component and a randomized trial testing its effectiveness by: 
4. Producing the full WayToServe Plus in-service professional development component. 
5. Conducting a randomized trial with 180 alcohol premises in California, New Mexico and Washington State 
evaluating effectiveness of WayToServe Plus on refusal of sales to PiP (primary outcome). 
H: Compared to premises in WayToServe RBS training only group, premises assigned to receive WayToServe RBS 
training plus WayToServe Plus will have higher rates of refusing PiP at posttest. 
 
The research and product is innovative. WayToServe Plus will be the first in-service professional development 
product for alcohol servers. The market is large and growing: 36 states require/incentivize RBS training and, starting in 
2022, California will require RBS training. WayToServe was licensed by Klein Buendel, Inc. (KB), U. of New Mexico (UNM), 
and Pacific Institute for Research and Evaluation (PIRE) to Wedge Communications LLC (WEDGE) that has sold over 
97,000 trainings in California, New Mexico, Washington State, Texas, and California. The WayToServe Facebook page, 
followed by over 20,000 alcohol servers who have completed the WayToServe training 
(http://www.facebook.com/WayToServe/), provides an innovative platform for the professional development of trained 
alcohol servers. WayToServe Plus will increase the training’s effectiveness and commercial advantage for servers and 
corporate clients. 
 
The WayToServe team is led by Dr. David Buller and Gill Woodall, Multiple Principal Investigators, from KB. Co-
Investigators are Dr. Gary Cutter (as an expert vendor) from KB and Dr. Bob Saltz from PIRE. WEDGE will provide access 
to WayToServe training to recruit alcohol servers and to WayToServe Facebook page to create, deliver, and 
commercialize WayToServe Plus (see Section 9A). 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
3. RESEARCH STRATEGY 
3A. Significance 
3A.1. Effectiveness of Responsible Beverage Service (RBS) Training 
The research is significant because it may improve an effective intervention for preventing driving while intoxicated 
(DWI), responsible beverage service (RBS) training for alcohol servers. In 2018, there were 39,404 deaths from motor 
vehicle crashes (MVC) and 4.5 million people injured, costing $445.6 billion.68 Effects of alcohol on driving impairment 
are well-established,69-73 with 10,511 people killed in alcohol-impaired MVCs (29% of MVC deaths) in 2018.2 Policy 
approaches to preventing alcohol-impaired driving have commonly focused on direct deterrence of drinkers who 
become impaired driving.74 An alternative to direct deterrence of drivers is marketplace policies to prevent access to 
alcohol by persons already intoxicated. Many states have laws prohibiting sales of alcohol to visibly-intoxicated 
customers to reduce DWI and other harms.9  
 
Another marketplace intervention that complements laws prohibiting sales to intoxicated patrons is RBS training 
that aims to instruct servers how to prevent intoxication by teaching drink-counting techniques, recognizing signs of 
intoxication, and strategies to refuse alcohol sales. Laws and training on RBS in the alcohol market is an environmental 
intervention to decrease opportunities for risky behavior,8 i.e., reducing access to alcohol, consistent with a nurturing 
environments perspective75 emphasizing harm reduction. They complement deterrence aimed at drivers with a targeted 
restriction on alcohol accessibility (i.e., refusal to sell at the specific time and place where the risk is greatest) that does 
not depend on decision-making by impaired persons. They can be applied to all premises, do not depend on the house 
policies of individual licensees to refuse sales, and reduce the ability of intoxicated customers to “shop around” to find 
premises that will serve.  
 
Research on RBS training presents a mixed picture. Some studies have failed to show effectiveness of some RBS 
training.76 However, reviews in 2000 and 2001 concluded that it can prevent alcohol over-service,77 with strong 
management support,1 and lower levels of apparent intoxication have been observed in premises in mandatory RBS 
training states.10 A Cochrane meta-analysis7 reported positive and negative impact. Despite questions about further 
research on RBS training,78 there are reasons to continue it. First, positive outcomes of RBS have been reported.7,79-81 
Second, methodological problems limit evidence (e.g., lack of randomized trials, clear outcome variables, and effect size 
reporting; inconsistent/incomplete data on training implementation/ fidelity),1,7,77 so we cannot be certain about its 
success. Third, most past research tested live training, despite the predominance of online technologies, which can 
improve training quality/fidelity (in-person trainers’ instruction can vary7) and lower cost. Also, multi-media features can 
present engaging online learning activities and immersive simulations, potentially making RBS effects more predictable. 
Our team showed that an online RBS training, WayToServe®, was effective in two randomized trials in onsite (bars and 
restaurants) and off-site (package stores) premises11 (see Box 1), which provides scientific premise for this SBIR research. 
Fourth, management commitment to RBS (which is not always present) can affect servers’ adherence to RBS principles 
taught in training.1,7 This finding provides a rationale for adding an in-service professional development component that 
follows on our WayToServe training to counter management disinterest or outright resistance.  
 
RBS training laws across U.S. states are highly variable: 23% of serving staff are in mandatory alcohol server training 
states; 58% in voluntary incentivized server training states; and 19% in states with no server training laws.82 In 2022, 
California will require alcohol server training for over 800,000 servers. One study10 found lower levels of apparent 
intoxication in premises in mandatory training states. With variable policies, uptake of RBS training is less than optimal. 
Our goal is to improve our effective online server training, WayToServe. 
3A.2. Theoretical Approach to In-service Professional Development for RBS  
The proposed research is significant because theory-based professional development delivered over social media will 
motivate servers to implement RBS, cope with barriers, and be part of a professional community of servers that supports 
one another and community safety. Diffusion of innovation theory (DIT)29,83 is the framework for improving 
WayToServe with in-service contact aimed at instilling professionalism. In DIT, prevention innovations like RBS must be 
implemented to be effective. Implementation is facilitated by making it compatible with values/needs of servers, 
managers, and owners, simple to use, easily accessible to try, and have observable outcomes (e.g., helping use RBS skills 
and overcome barriers). Compatibility and simplicity of RBS will be stressed in its content and simplicity, trialability, and 
observability by using popular social media. DIT, along with social network principles,29 also predicts how social media 
content can influence. Elevated audience involvement in social media increases dissemination and impact84 and 
influence involves delivering carefully-crafted content by change agents (e.g., experts) and spreading it among the 
community by opinion leaders (i.e., knowledgeable peers) to stimulate collective action29 that reduces uncertainty.30-32 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
Content shared in social media results in opinions/sentiments becoming dominant by interactions among users34-36 
rather than a simple accretion of messages from isolated users.37 Attitudes emerge from a group dynamic, not mere 
exposure, and can breed collective action through a process of social comparison/ identity.29,38-40 Users compare 
themselves with group members41 based on social categories and roles (e.g., female, friend, caring person, server) and 
conform to avoid uncertainty,42 creating collective identity and stabilizing behavior change.38,39 Influence of social media 
is also explained by transportation theory (TT)85,86 and research on narratives. People’s behavior is governed by stories 
about their lives87,88 and risk behavior is explained by cultural narratives (e.g., young age excuses risky alcohol use). 
Comments/testimonials that contain personal stories can be more powerful than didactic messages and expert advice.89 
People transported into narratives often change their beliefs based on information, claims, or events depicted90 that 
conform to existing cognitive schemas91 and make narratives seem real. Persons identify with characters in a story, 
increasing influence.92,93 Narratives can shift normative beliefs about risks and responses to it.94-101  
 
Scientific premise is provided by literature showing training, in-service contact, and certification has improved 
professionalism of community members who deliver prevention services. In-service improved tobacco control and 
maternal health services.12-14 In healthcare, continuous training improved hand-washing to prevent infection and 
barcode technology to reduce medication errors.15 In-service contact for RBS is intended to improve servers’ 
professionalism by a) motivating them to implement RBS skills in the face of barriers (e.g., pressure to sell, management 
disinterest or resistance, and customers’ attempts to be served), b) providing support for RBS from a “community” of 
servers, especially for those in small or unsupportive premises, and c) preventing degradation of skills over time by self-
enrolling into booster training. Based on our theoretical approach and preliminary data (see Boxes 1 and 2), in-service 
professional development will be delivered over social media (i.e., Facebook, Instagram, and YouTube). It will reinforce 
WayToServe training and invite user-generated content from servers that supports RBS and creates a sense of 
“community” among servers. We will encourage stories from other servers, have them share their RBS tips and tricks, 
and incorporate testimonials and videos/photos to capitalize on interpersonal/interpretive processes that sustain 
change.  
3A.3. Summary of Significance 
The proposed research is highly significant. Alcohol-impaired driving is very high-risk and RBS training has shown 
some effectiveness at preventing it. The SBIR research will show if adding continuing instruction and developing 
professionalism among servers improves it. Fast-track SBIR research is justified because: 
 
Acceptability, feasibility, and effectiveness of online RBS training was validated in prior trials (see Box 1). 
 
Preliminary research on WayToServe-trained alcohol servers (see Box 2) confirmed need for and interest in an in-
service professional development component, informed plans for in-service topics and learning activities, and 
verified feasibility of delivering it through WayToServe Facebook page (see Box 3).  
 
Successful protocols are available from our prior RBS trials for recruitment of premises and servers, implementation 
of WayToServe training and WayToServe Plus on social media, assessment of RBS outcomes with pseudo-patrons, 
and analysis of data collected in RBS trials.  
 
A feasible prototype for WayToServe Plus and plans for full production and evaluation of this component developed 
in Phase I can be immediately applied in Phase II. 
3B. Innovation 
The major innovation in this SBIR Fast-track research is the creation of an in-service professional development 
component, WayToServe Plus, to improve effectiveness of the evidence-based WayToServe RBS training. 
 
WayToServe Plus will be the first RBS training that includes in-service contact promoting professional development 
of alcohol servers. Commercially-available RBS training, including WayToServe, provides a single training. States 
typically require that servers be retrained every 2-5 years.  
 
WayToServe Plus will fill a large gap in RBS training required/incentivized by over half of U.S. states by providing in-
service training and creating a community of servers that support implementation and habituation of RBS and 
professionalism among servers trained by WayToServe. 
Our experience with WayToServe training and its Facebook page (a community of over 20,000 trained servers; Box 3) 
and preliminary research (see Box 2) suggests that WayToServe-trained servers will engage with and benefit from in-
service professional development. In Phase I, we will further confirm feasibility of WayToServe Plus and develop content, 
before full development and testing in Phase II. In 2019, most adults used social media (72%; 90% aged 18-29; 82% aged 
30-49102) for information and peer connections103 that can influence.104,105 Despite some decline in user base,106-108 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
Facebook reaches a large majority of adults and is among the most popular social media (69% used it in 2019 [74% 
daily]; 79% aged 18-29; 79% aged 30-49),28 including for alcohol servers (see Box 2). Facebook’s private group feature 
provides control of treatment presentation to prevent contamination when testing effectiveness of WayToServe Plus. 
We will also link to in-service content posted on Instagram and YouTube, two other highly popular social media for 
alcohol servers (see Box 2).  
 
If effective, WayToServe Plus will have high impact on a popular state DWI prevention strategy and provide a 
commercial advantage for WayToServe training. The market is large and growing (36 states require/incentives it; 
California will require RBS training in 2022). WayToServe Plus will differentiate the WayToServe training from its 
competitors, such as eTIPS and ServSafe, as the only training with in-service professional development. For corporate 
clients, its value-added content will ensure investment in the training maintains RBS, increasing group sales. For alcohol 
servers, it will provide a professional community and improve retention for renewal trainings. Under its license 
agreement for WayToServe training, Wedge Communications LLC (WEDGE) will immediately commercialize WayToServe 
Plus (see letter of support in Section 9A) in California, New Mexico, Washington State, Texas, and California, where it 
has sold over 97,000 trainings since 2012. WEDGE recently upgraded WayToServe to improve operability on 
smartphones and tablets. 
Box 1: Preliminary Studies on WayToServe RBS Training
With R01 grants from NIAAA (AA14982, AA016606 PI: Woodall), we developed an 
effective online RBS training, WayToServe (https://WayToServe.org/).11 It is the 
only commercially-available RBS training tested in randomized trials. WayToServe is 
compliant with alcohol server training requirements in New Mexico, Texas, and 
Washington State, where it is a state-approved RBS training in English and Spanish.  

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
WayToServe RBS Training: WayToServe teaches information and skills for RBS as determined by state regulations 
and formative research in 6 sequential modules: 1) Alcohol laws - What one can and must do legally as a server and 
penalties for non-compliance. 2) Preventing service to minors – Methods to identify underage patrons, e.g., checking 
IDs. 3) Facts about alcohol - Effects of alcohol, how to recognize an intoxicated patron, and standard drink and blood 
alcohol concentration (BAC). 4) The intoxicated patron – How to take an active 
role in refusing alcohol service to intoxicated patrons to prevent DWI and other 
harms. 5) Responsible alcohol sales practices - SERVE system, a unique drink 
counting method to keep patrons under 0.08 BAC driving limit. 6) Alcohol and 
pregnancy - Information about Fetal Alcohol Spectrum Disorders (FASD). Learning 
activities were based on principles of adult learning (i.e., learning with application 
and feedback109-111 and Social Cognitive Theory (SCT; i.e., learning via modeling, 
practice, and feedback; see Figure 1).112,113 States require servers view all content 
and learning activities, and pass end-of-module quizzes and final exam (at least 
80% correct) to receive state certificate to serve alcohol. It takes about 3 hours to 
complete the training. 
 
Evaluation of WayToServe Training: Two randomized trials were performed, 
with onsite (n=155) and off-site (n=158) alcohol premises in New Mexico, 
comparing WayToServe to usual and customary training (UC) provided by live 
trainers in 2 (training: WayToServe vs. UC) x 4 (time: baseline, immediate post-training, 6-month and one-year follow-
up) design. Primary outcome was refusal of service to “pseudo-patron” buyers114-119 who acted apparently-intoxicated 
(refusal to underage-appearing pseudo-patrons was assessed in Off-site Trial, too). Onsite Trial11 had a significant 
main effect for training (F (1,264)=5.55, p=.019) and training by time interaction (F(3,792)=2.88, p=0.35) on refusal 
rates. Improvements for WayToServe compared to UC occurred at immediate 
post-training (t(288)=3.26, p<0.001) and 1-year (t(272)-1.74, p=0.002; see Figure 
2). In Off-site Trial, there were a significant effects for training (F(2,1311)=2.87, 
p=.05) and training by time interaction (F(3,1311)=2.72, p=.04), and trend toward 
a difference at 1-year (WayToServe M=76%, UC M=68%; t(292)=1.42, p=.07). 
WayToServe improved refusals of underage pseudo-patrons, too (training 
F(1,1024)= 19.19, p=.0001; training X time F(3,1024)=2.69, p=.045; 6-months: 
t(308)=2.58, p=.005; 1-year (t(101)=3.02, p=.001) (see Figure 3).  
 
Scientific Premise: These data show that WayToServe training was effective at 
improving RBS practices. Further, its usability (M=63.73 out of 70) and servers’ RBS 
self-efficacy (M=45.94 out of 50) remained high in dissemination. But, the Off-site 
Trial suggested servers will benefit from more RBS support/training. This is the 
goal of the proposed in-service professional development component. 
3C. Research Design and Methods 
The SBIR Fast-track will be completed in 3 years (see timeline in Form F, Section 2.7). Phase I protocols will be developed 
with External Advisory Board (EAB) in Month 1. Owners/managers and alcohol servers from onsite alcohol premises will 
give input on in-service professional development in Month 2 in focus group/interviews/survey. A prototype 
WayToServe Plus will be produced in Months 3-4. Acceptability, feasibility, and engagement with the prototype will be 
tested with alcohol servers in focus groups in Months 5-6 and in a field pilot test in Months 7-8. Phase I results will be 
reviewed by EAB in Month 9 for feasibility of full production and evaluation. In SBIR Phase II research, a specifications 
document approved by EAB will guide full production of WayToServe Plus (if deemed feasible) in Months 10-17; EAB will 
review full WayToServe Plus and randomized trial plans. WayToServe Plus will be tested for effectiveness in Months 15-
36 in a randomized two-group design. All protocols will be approved by the Western Institutional Review Board (WIRB).  
3C.1. Target Populations, Recruiting, and Inclusion/Exclusion Criteria 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
Owners/managers and alcohol servers from licensed premises selling alcohol by the drink (e.g., bars and restaurants) in 
California, New Mexico and 
Washington State will participate (see 
Figure 4 for inclusion/ exclusion 
criteria). Onsite premises will be 
enrolled because Washington State 
requires RBS training only for onsite 
sales. For Phase I, alcohol servers will 
be selected at random from the roster 
of WayToServe trainees over the 
preceding 12 months to obtain newly-
trained servers and those who had 
time to implement RBS. In 2019, 
WayToServe averaged 45 trainees per 
day (~1,360 per month). In our preliminary survey (see Box 2), 70% of WayToServe-trained servers were willing to test 
WayToServe Plus. Thus, recruitment of servers is feasible.  
3C.2. External Advisory Board (Month 1-9; Specific Aim 1) 
The EAB will be chaired by Dr. Saltz and include Dr. Sherry Pagoto (an expert in using social media for disease 
prevention), and Mr. David Mosher (an alcohol industry representative). We will recruit an owner/manager and alcohol 
server who meet the inclusion/exclusion criteria above for the EAB. EAB will meet 6 times: 
1. Month 1: Review plans and protocols for the SBIR Phase I research in Month 1,  
2. Month 4: Review specifications document and prototype of the WayToServe Plus. 
3. Month 9: Review Phase I results and advise on feasibility of testing WayToServe Plus in Phase II. 
4. Month 13: Review all protocols for randomized trial in Phase II. 
5. Month 17: Review fully-produced WayToServe Plus in-service professional development component.  
6. Month 36: Review results of randomized trial and suggest final improvements to WayToServe Plus.  
Milestones & Deliverables: Milestone 1: Approval of Phase I Research Procedures: In Month 1, the EAB will review and 
approve the Phase I research procedures. Deliverable 1: Approved Study Protocol. 
 
 
 
 
 
 
 
 
 
 
 
 
 
Box 2: Survey of WayToServe-trained Alcohol Servers on WayToServe Plus Concept

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
In 2019-20, we surveyed 100 alcohol servers (n=50 New Mexico, 50 Washington State) who completed WayToServe
training in past year (71% female; 36% ethnic minority; mean [M] age=33.6; 55% onsite; 48% were serving alcohol <1 
year [M=4.6]) on their need for and interest in in-service professional development for RBS. Servers’ most popular 
social media platform was Facebook at all ages, with most servers having an account and checking it several times a 
day (see Table 1). Also popular were Instagram and 
YouTube. Use of these 3 social media did not differ by 
age, gender, or type of premises (p>0.05), but 
YouTube is used most by servers with 2-5 years of 
experience (0-1 years 47.9%, 2-5 70.4%, 6+ 21.0%, 
p=0.004). Of the popular platforms, YouTube and 
Instagram have higher credibility than Facebook. 
 
About a quarter of servers were not completely 
confident they can use RBS practices from 
WayToServe: Take steps to prevent intoxicated 
patron from driving (28.2% very sure), count drinks 
to prevent intoxication (30.9%), take other steps to 
prevent intoxication (28.6%; white 42.6%, Hispanic 
8.0%, other 27.3%; p=0.044), recognize when patron 
is intoxicated (21.2%), refuse service to intoxicated 
patron (11.1%), and use methods from WayToServe (21.2%). 74.2% of servers have refused alcohol service, but fewer 
younger (19-24 61.8%, 25-39 90.6%, 40+ 77.3; p=0.036), Hispanic (white 82.7%, Hispanic 72.0%, other 50.0%; 
p=0.023), and new servers (0-1 years 60.4%, 2-5 80.8%, 6+ 100%; p<0.001) had done so (nearly all are can check IDs – 
91.9% verify validity of ID; 97.0% verify age). A number of servers do not feel they have a lot of support of managers 
and servers for RBS (see Table 2) more in Washington State (managers: counting drinks: WA 36.6%, NM 13.2%, 
p=0.011; refusing service to intoxicated customers: WA 17.1%, NM 4.8%, p=0.048; cutting off customers close to 
intoxication: WA 32.6%, NM 14.0%, p=0.038; other servers: checking IDs: WA 19.1%, NM 7.0%, p<0.046; refusing 
service to intoxicated patrons: WA 24.4%, NM 8.9%, p=0.035).  
 
WayToServe Plus appealed to WayToServe-trained servers: 73.9% were interested in ongoing 
information/activities to keep up-to-date and use RBS methods (20.6% a little, 26.1% somewhat, 27.2% very) and 
89.4% felt they would benefit (27.4% a little, 26.3% 
some, 35.8% a lot). Older servers had highest 
interest (19-24 59.4% 25-39 72.4%, 40+ 91.7% at 
least a little interested; p<0.001) and benefit (19-24 
81.8%, 25-39 90.0%, 40+ 100% at least a little 
benefit; p<0.001). 55% of servers would join 
WayToServe Plus Facebook group (20.0% a little, 
19.0% some, 16.0% a lot; no age difference p=0.500) 
and visit it at least weekly (27.8% once a week, 
15.2% several times a week, 8.8% daily). At least half 
of servers would be interested in and/or use various 
topics/features (see Table 3). Most servers said 
joining WayToServe Plus would make them feel like 
they were in a professional community (76.5%). 
 
Most WayToServe-trained servers (70.0%) would take part in testing in-service component. Some had privacy 
concerns (48.9%: WA 59.2%, NM 36.6%,p=0.002) but 63% trusted Facebook to protect personal data (less trust by 
new servers: 0-1 years 51.2%, 2-5 76.0%, 6+ 43.8%, p=0.016). 
 
Scientific Premise: The results supported the need for in-service professional development and the WayToServe 
Plus component delivered through the WayToServe Facebook page.  
 
A substantial group of alcohol servers trained by WayToServe have room to improve their self-efficacy for RBS 
practices to reduce intoxication, especially younger and newer servers.  
 
A number of servers also do not have strong support from management and other servers for RBS and support 
varies by state validating our plans to test the WayToServe Plus in two states.

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
 
Most servers were interested in getting in-service information and felt it would be beneficial and over half would 
join it through a Facebook group. Younger servers may need extra effort to recruit them.  
 
WayToServe Facebook page is a good platform to deliver WayToServe Plus. Facebook is the most popular social 
media. We also will post on Instagram and YouTube, given their popularity and credibility.  
 
We will emphasize interactive activities, notable, unusual, or humorous stories from servers, and tips and tricks 
for using RBS practices, and message features to encourage engagement, sharing, and comments.  
 
Ways to protect privacy of server-generated content on WayToServe Plus are needed. 
 
We should be able to recruit many servers to participate in the proposed SBIR research.
3C.3. Phase I: Formative Research on RBS Professional Development (Month 2; Specific Aim 1) 
 
3C.3.1. Semi-Structured Interviews with Alcohol Premise Owners/Managers: Ten owners/managers of onsite 
alcohol premises in California, New Mexico (n=5) and Washington State (n=5) will participate in 1-hour semi-structured 
interviews performed by Drs. Buller and Woodall, using KB’s video conferencing software, Zoom (compensation=$75). 
Before the interviews, owners/managers will be sent the URL for the WayToServe online RBS training and instructed to 
review it. Owners/managers will discuss RBS policies at their premises, their support for RBS methods, perceived 
importance of RBS, quality of RBS methods implemented by their servers, need for ongoing training and support for RBS 
among their servers, and suggested content for in-service professional development for alcohol servers.  
 
3C.3.2. Focus Groups with Alcohol Servers: 2-hour virtual focus groups with approximately 22 alcohol servers from 
onsite premises in New Mexico (n=~11) and Washington State (n=~11) will be led by the Project Coordinator trained in 
focus group methods, using KB’s video conferencing software, Zoom (compensation= $75). Servers will be recruited at 
random from the roster of WayToServe trainees in the past year. Servers will describe content and features desired for 
in-service professional development in RBS methods, barriers to implementing RBS and means of overcoming them, and 
potential utility of in-service professional development for premises and servers. They will discuss support received from 
premises owners/managers and other alcohol servers for RBS and its adequacy, and their experience with RBS practices, 
especially after receiving state-mandated RBS training. They will be asked if they have ever refused a sale, how confident 
they were in doing so, and techniques used to successfully/unsuccessfully refuse. Servers will be asked if they are willing 
to share RBS experiences or provide feedback on RBS actions to other servers on the WayToServe Facebook page.  
Milestones & Deliverables: Milestone 2: Development of the Content Outline: Investigators and KB media developers 
will review results of our preliminary survey (see Box 2) and summaries of the formative interview and focus group 
results and create an initial specifications document describing all aspects of the professional development component 
to guide production of the prototype WayToServe Plus, i.e., detailed content outline and features desired by owners, 
managers, and servers and feasible through Facebook, Instagram, and YouTube. It will also describe technical 
requirements and programming plans for the component. The EAB will review and give input on the initial specifications 
document. Deliverable 2: Specifications Document.  
3C.3.3 Survey of Alcohol Servers 
In addition to the above focus groups, a survey will be distributed to alcohol servers at on-site premises in New Mexico 
(n=~15) and Washington State (n=~15). Servers will be recruited at random from the roster of WayToServe trainees in 
the past year. Servers will describe content and features desired for in-service professional development in RBS 
methods, barriers to implementing RBS and means of overcoming them, and potential utility of in-service professional 
development for premises and servers. They will discuss support received from premises owners/managers and other 
alcohol servers for RBS and its adequacy, and their experience with RBS practices, especially after receiving state-
mandated RBS training. They will be asked if they have ever refused a sale, how confident they were in doing so, and 
techniques used to successfully/unsuccessfully refuse. Servers will be asked if they are willing to share RBS experiences 
or provide feedback on RBS actions to other servers on the WayToServe Facebook page.  
 
This survey is meant to supplement the focus group data in light of the COVID-19 Omicron variant surge, not replace it.  
3C.4. Phase I: Production of the Prototype WayToServe Plus (Months 3-4; Specific Aim 2) 
A prototype of WayToServe Plus in-service professional development component will be produced by KB’s media 
developers (see Section 8A). The goal of in-service component is to raise servers’ professionalism by:  
1. increasing confidence and motivation of WayToServe-trained servers to implement RBS methods, with attention to 
ways of overcoming common barriers such as pressure to sell from managers, management disinterest or 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
resistance, and customers’ attempts to continue being served;  
2. creating a professional community of servers on WayToServe Facebook page that provides support to fellow servers 
for implementing RBS actions and serves as resources for advice/strategies to implement RBS (e.g., tips and tricks), 
which should be especially useful for new servers, servers in small or unsupportive premises and in the face of 
turnover in premises management and servers, by sharing their personal RBS experiences through user-generated 
comments and sharing of the posts. 
3. preventing natural degradation of RBS skills and motivation over time by providing refresher instruction through 
text, interactive features, and videos demonstrating RBS techniques, which are based on principles of DIT (e.g., 
compatibility, simplicity, trialability, and observability; see 3A.2) and SCT (i.e., modeling). 
Based on formative research (see 3C.3 and Box 2), We will author a library of 50 Facebook posts from which a 1-month 
prototype will be created (see below). Posts will provide help for RBS suggested by owners/ managers, present ways to 
overcome barriers to implement RBS and refuse sales, describe success stories from servers; and stimulate supportive 
comments/feedback on RBS from other servers. To achieve high visibility on users’ Facebook feed, given its revised 
newsfeed algorithm, we will post multiple times per week and format posts to increase user engagement (e.g., polls; 
true/false; activities; “fun” content).  
 
Investigators and media developers will use agile, iterative production for WayToServe Plus and methods used by 
Drs. Buller and Pagoto (Consultant) to create an effective social media campaign in a prior studies (see Box 4). Posts will 
be based initially on instructional goals, DIT and SCT principles, and formative research and, later, on usability testing 
and field pilot test results (see 3C.5) and be relatable, positive, entertaining, and at 4th grade reading level. Posts will be 
created using message grids, where each message is listed, along with theoretic principles operationalized and 
suggestions for graphics/weblinks to related content (e.g., videos, interactive activities, or external sources [e.g., news]). 
Investigators will create initial scripts for interactive exercises, based on WayToServe training, and short videos depicting 
RBS methods (e.g., ID checking, spotting intoxicated patrons, refusing sales). Interactive simulations will teach skills by 
presenting challenges (e.g., checking IDs; refusing sales to intoxicated patron; and counting drinks), making a choice that 
applies the skill (e.g., decide if ID is valid or not or to serve alcohol or not), and receiving corrective feedback. Media 
developers will add the multimedia elements to posts and storyboard activity/video scripts. Investigators will review all 
posts and scripts/ storyboards, making final adjustments, and media developers will produce interactive activities and 
videos. Features to elicit user-generated content will be included in posts (e.g., pose a common situation and ask for 
example RBS strategies, solicit stories about “most outrageous manager” or “drunkest customer,” or contest for “server 
of the month”) to create sense of “community” among alcohol servers. WayToServe Plus will be in English because a) 
nearly all social media content in the United States is in English (1.9%)120 and b) most servers elect to complete 
WayToServe training in English (98.4% in 2019). We will incorporate Spanish posts once success of WayToServe Plus is 
established in this SBIR research.  
 
WayToServe Plus posts (n=12) will be selected for a 1-month prototype for the field test (see 3C.5.2) that adequately 
demonstrate various formats of posts (text, graphics, interactive activities, and videos) and the key professional 
development goals. Media developers will produce a Facebook private group with the look of the WayToServe 
Facebook page where the WayToServe Plus posts will be posted. Interactive activities, graphics, and videos will be 
posted on a custom webpage, Instagram, or YouTube, accessed via links in the Facebook posts. Some activities will 
function best on a webpage; Instagram and YouTube are popular social media with alcohol servers (see Box 2). The 
Project Coordinator will serve as the Community Manager and schedule posts, monitor feed, and respond to 
inappropriate comments. The EAB will review the prototype. 
Milestones & Deliverables: Milestone 3: Development of WayToServe Plus Prototype: The WayToServe Plus prototype 
will contain a library of Facebook posts with text, graphics/photos, interactive activities, and videos, schedule for a 1-
month field test, and updated specifications document. It will give a good sense of how the in-service component will 
operate and how much servers will engage with and benefit from it. Deliverable 3: Revised Specifications Document. 
Deliverable 4: WayToServe Plus Library of Facebook Posts. 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
Box 3: WayToServe Facebook Page
WayToServe Facebook page is a marketing platform, with 20,323 likes and 
20,246 followers as of August 27, 2020. WEDGE posts informative, entertaining 
content on alcohol service (approx. 7 per week), including ads for WayToServe 
training and RBS information (e.g., BAC, mocktail recipes, holiday alcohol 
safety). Most posts are delivered to all states, with some geo-targeted locally. 
In August, 2020, 259,667 users saw a posting from WayToServe on their 
Facebook newsfeed (reach), 1913 users engaged with these WayToServe 
Facebook posts (i.e., liked, shared, or commented on a post) (engagement), 
and 437 users saw the last 5 Facebook posts from WayToServe. This use 
occurred without commitment or compensation; we will confirm that use will 
be higher by servers who enroll in a formal study testing the in-service 
component. 
3C.5. Phase I: Evaluation of Prototype WayToServe Plus (Months 5-8; Specific Aim 2) 
The prototype WayToServe Plus component will be evaluated for feasibility, acceptability, usability, and engagement 
through focus groups with alcohol servers in Months 5-6 and a field test with alcohol servers in Months 7-8 (servers who 
participated in formative research [see 3C.3] will be ineligible).  
 
3C.5.1. Focus Groups on Acceptability, Feasibility, and Usability (Months 5-6): Twenty alcohol servers will be 
recruited at random from the roster of WayToServe trainees (n=10 in New Mexico and 10 in Washington State; 
compensation=$75) to participate in focus group discussions on the acceptability, feasibility, and utility of the prototype 
WayToServe Plus, led by the Project Coordinator (10 usability testers can identify 95% of problems121-123). They will be 
presented with a description of WayToServe Plus and review the library of Facebook posts. For each post, servers will 
identify the main message (to test comprehension), indicate if it is appropriate and acceptable for alcohol servers, their 
premises, and their experience deploying RBS methods, report if the information will be useful to them and other 
alcohol servers, say whether they are likely to view, comment, or share the post, or contribute their own content on RBS 
methods to the Facebook page, and note what would keep them from engaging with posts (i.e., barriers to use). 
Investigators and KB production staff will revise the WayToServe Plus based on this feedback and before field testing 
(see 3C.5.2). 
 
3C.5.2. Field Pilot Test of Prototype WayToServe Plus (Months 7-8): A field pilot test of a 1-month prototype of 
WayToServe Plus will be conducted with 100 alcohol servers (n=50 New Mexico; n=50 Washington State; 
compensation=$100) recruited when completing WayToServe training and randomized to WayToServe Plus (n=50) or 
untreated control (n=50). Servers in WayToServe Plus group will “follow” a WayToServe Plus Facebook private group, 
administered by the Project Coordinator that has the look of the WayToServe Facebook page but messages will be 
viewable only to group members. Project Coordinator will post 3 WayToServe Plus posts per week for 4 weeks (n=12 
posts), along with normal posts by WEDGE’s marketing staff. Servers’ engagement with WayToServe Plus will be 
recorded – number of times posts are viewed, reacted to, and commented on, and user-generated content posted such 
as personal experiences with refusing sales to intoxicated patrons or entry by minors. After 4 weeks, servers will 
complete a posttest in KB’s Qualtrics online survey software assessing acceptability/appropriateness of WayToServe Plus 
in-service component for servers and premises, and their experience using RBS methods, usability of WayToServe Plus 
on validated System Usability Scale (SUS),124-126 utility of in-service posts, and perceived self-efficacy and response-
efficacy (for keeping community safe) for RBS methods. Servers will be asked how willing they are to use WayToServe 
Plus in the future and how they would improve it and make it more engaging.  
 
Ten alcohol servers in the field test sample (n=5 New Mexico; 5 Washington State; compensation=$50) will be 
randomly selected for a semi-structured interview on reactions to WayToServe Plus prototype. The Project Coordinator, 
using video conferencing, will probe for in-depth input on whether the in-service component was appropriate, 
acceptable, and useful for themselves, their premises, and their RBS methods, what made them engage with the posts 
or not, and how the in-service component could be improved and made more engaging.   
Milestones and Deliverables: Milestone 4: Evaluation of the Prototype: Reports will be prepared on the usability and 
field test findings in Month 9 that describe overall usability, valued parts of the prototype, ways to improve the in-
service component in Phase II (i.e., solutions to remaining problems planned for Phase II production), engagement with 
the prototype, and barriers/willingness to use WayToServe Plus. KB media developers will update the specifications 
document for full production of WayToServe Plus in Phase II. For planning the evaluation of WayToServe Plus in Phase II, 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
reports will summarize recruitment and follow-up rates in the field test. Deliverable 5: Evaluation Reports; Deliverable 6: 
Revised Specifications Document. 
Box 4: Previous Technology Intervention Development and Testing
Drs. Buller and Woodall and KB media developers are experienced with technology interventions, including on 
adolescent smoking,127 dietary change,128 risky drinking, 129 risky sexual behavior,130 and HPV vaccine uptake,131 as well 
as the WayToServe training. Scientific premise for delivering WayToServe Plus over social media is provided by past 
research by Drs. Buller and Pagoto (Consultant). They tested a social media campaign to prevent indoor tanning (IT) 
by adolescent girls in a randomized trial (R01CA192652; see 
Figure 6).132 Mothers (n=869) with teen daughters were 
enrolled. Relevant to the SBIR Fast-track research, mothers 
received either intervention or control social media 
campaign by joining Facebook private groups, to control 
treatment condition and prevent contamination. IT 
messages were posted 3-4 times per week over 12 months 
(~185 posts each; 87% stayed in groups for 12 months). 
Follow-up was high (72.4% at 12-months; 78.9% at 18-
months). At 12-months, the social media campaign reduced 
mothers permissive toward daughters’ IT (p<0.05) and IT 
intentions (p<0.05) and increased communication with 
daughters about avoiding IT (p<0.05). Dr. Pagoto used social media in 3 studies of successful social media weight loss 
interventions (R21DK098556, K24HL124366, KL2TR000160, U01HL105268)133-135 and 5 ongoing studies of weight loss 
interventions (R01DK103944-01A1, R01CA192652, R01HL122302, K24HL124366, R21CA226133). Drs. Buller and 
Pagoto are collaborating on a family-based Facebook intervention on skin cancer prevention (R01CA221854; S. 
Manne, PI). 
 
Scientific Premise: We created theory-based social media campaigns that improved health behaviors. We have 
protocols to develop social media messages, recruit and retained multi-ethnic adults, and deliver, track, and analyze 
social media messages using Facebook private groups to achieve experimental control.  The SBIR Fast-track research 
will leverage this experience to create, deliver, and test WayToServe Plus.
3C.6. Phase I: Data Management and Analysis  
Qualitative analysis of interviews, focus groups, and survey data will rely on constant comparative analysis136 and 
methods by Krueger and Casey.137 Atlas-ti© software will be used to assign codes to categorize themes and opinions. To 
identify problems that impact >10% of users in usability testing138,139 with a 90% chance of detection (log(1-.90)/log(1-
.10)), 21 participants are needed. For field pilot test, we will enroll 25 servers per state per condition (n=50 WayToServe 
Plus; n=50 control). Control sample will allow us to estimate follow-up rates in both conditions for Phase II randomized 
trial. Data from field pilot test of prototype WayToServe Plus will be stored in KB’s secure servers, with data 
management and analysis supervised by Dr. Cutter, Project Biostatistician and Ms. Lucia Liu, Biostatistical Manager, 
using quality assurance protocols and SAS software. Servers’ pre-post change in theoretic mediators of RBS (i.e., self- 
and response efficacy) will be compared between WayToServe Plus and control, using Fisher’s exact test and t-tests 
(intention-to-treat), to show potential effectiveness (for proportions of 35% to 50%, differences of about 25% can be 
detected with 80% power; for continuous outcomes, power is >80% for effects sizes >0.55 sd). Comparisons on server 
demographics and premises characteristics will be done using χ2, t-tests, and correlations. Usability of (ratings on SUS) 
and engagement with in-service component (i.e., viewing; commenting; and contributing posts), evaluations of its 
content, and barriers and willingness to use it will be collected and analyzed. Analyses will include descriptive statistics 
and regression analyses, using intention-to-treat analyses and sensitivity analyses adjusting for covariates and levels of 
adherence. All tests will be at 2-tailed p=0.05. 
3C.7. Appraisal of Feasibility of WayToServe Plus and Phase II Plan (Month 9; Specific Aim 3) 
The EAB will meet in Month 9, review Phase I findings, and advise on feasibility of WayToServe Plus. Feasibility is 
indicated by a) WayToServe Plus being acceptable and desirable to 75% of owners/managers and servers, b) 
WayToServe Plus specifications and prototype being developed, c) WayToServe Plus prototype being acceptable, usable 
(minimum 68 out of 100 on SUS67), and engaging (75% of users view and 50% react to a post [e.g., like; comment]), and 
d) WayToServe Plus influencing theoretic mediators (at least small effect sizes, Cohen’s d=0.2). They will provide input 
on specifications document, ways to improve WayToServe Plus and research design and implementation procedures for 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
evaluating WayToServe Plus in Phase II.  
Milestones and Deliverables: Milestone 5: Feasibility Assessment: The EAB will review Phase I outcomes and advise on 
feasibility of the full WayToServe Plus, its specification document, and research plans for SBIR Phase II research. A 
feasibility report will contain Phase I findings, decision on feasibility of WayToServe Plus and plans for Phase II research. 
Deliverable 7: Phase II Plan and Final Specifications Document.  
3C.8. Scientific Rigor and Limitations in Phase I  
State-of-the-art research methods, pre-defined protocols, diverse samples from two states, and established measures 
(e.g., SUS) will ensure scientific rigor of the SBIR Phase I research. Small samples are a limitation but typical for 
qualitative research; sample in field pilot test is moderate in size and provides adequate precision and will identify 
problems to create a usable in-service component. The control sample is designed to test for statistical differences in 
effect moderators, serve as a calibration sample to test acceptability of control condition, and estimate follow-up rates 
for Phase II randomized trial that will test WayToServe Plus. 
3C.9. Overview of Phase II Research and Timeline 
SBIR Phase II research will include full production of WayToServe Plus in-service professional development component 
(Months 10-17) and a randomized 2-group field trial testing its effectiveness (Months 15-36), in a 27-month timeline 
(see Section 2.7 in Form F). The target population (see 3C.1) is owners, managers, and alcohol servers working in state-
licensed premises for onsite sale of alcoholic beverages in California, New Mexico and Washington State. These two 
states require servers to complete state-approved RBS training such as WayToServe and be certified to serve alcoholic 
beverages to work as an alcohol server (and renew every 3 years in New Mexico or 5 years in Washington State). 
Investigators and media developers at Klein Buendel (KB) will produce the WayToServe Plus in-service component, using 
Phase I specifications, content outline, and prototype (see 3C.4 and 3C.5). The randomized trial will be performed, 
starting in Month 15, with intervention and assessment concluding in Month 32 and outcome analyses completed by 
Month 36. The EAB will meet in Month 13 to approve trial protocols, in Month 17 to review the WayToServe Plus in-
service component, and in Month 36 to review trial outcomes (see 3C.2). All Phase II procedures will be approved by the 
WIRB. Premises with participants in Phase I research will be ineligible for Phase II randomized trial. 
3C.10. Phase II: Production of WayToServe Plus (Months 10-17; Specific Aim 4) 
Investigators and media developers will author a library of ~160 Facebook posts for WayToServe Plus in-service 
component, planning to post 3 posts per week in a year (currently WEDGE posts 7 posts per week to WayToServe 
Facebook page, see Box 3), along with interactive activities and short videos. We will draft posts and script/storyboard 
activities/videos using methods from Phase I (see 3C.4). Prototype WayToServe Plus will be a starting point for the 
complete library of in-service posts (see 3C.4), with changes suggested by usability testing, server reactions/engagement 
in field pilot test, and EAB at the end of Phase I. To achieve high visibility on users’ Facebook feed, with its revised 
algorithm which downgrades posts that get low engagement, we will format posts to be engaging (e.g., polls; true/false; 
activities; “fun”). To increase relevance, posts will address barriers/facilitators and contain ethnically-diverse images 
from KB’s Real Health Photos stock art library that improves identification and impact.140 Frequency of posts will meet 
standards of repetition.141 Half of posts will be created before launch of WayToServe Plus; the other half during the 
intervention to leverage news items/current events for relevance. In our previous social media campaign, after 6 
months, participant engagement declined,142 indicating topic fatigue. We will monitor engagement/comments for 
evidence of fatigue, adjust post schedule, add videos/images for salience, and link to current events for 
engagement.84,143,144 Using these methods, 86% of mothers stayed in private groups in our past social media campaign 
(Box 4). EAB will review the library of posts, activities, and videos, suggesting final changes. 
Milestones and Deliverables: Milestone 6: Production of WayToServe Plus In-service Component. Deliverable 8: Fully-
produced WayToServe Plus. Deliverable 9: Randomized Trial Evaluation Plan. 
3C.11. Phase II: Experimental Design for Randomized Trial (Months 15-36; Specific Aim 5) 
We will test WayToServe Plus in-service component’s ability to increase refusal of service to intoxicated patrons in a 
randomized trial (see timeline in Section 2.7, Form F). A two-group randomized field trial (WayToServe training only 
[comparison control] v. WayToServe training plus WayToServe Plus in-service component [intervention]) with 2 
assessment rounds (Baseline and Posttest [12-months post-randomization]) will yield a 2 (treatment) x 2 (assessment 
time) factorial design, with 4 assessments per alcohol premises (2 per round; see Figure 7). State-licensed onsite alcohol 
premises (n=180) in California, New Mexico and Washington State will be randomly-selected from publicly-available 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
lists, stratified by location (metro areas [Albuquerque, Seattle; n=110] v. small towns/ rural areas [n=50] within 1-day 
drive of metro areas for feasible assessment) and assessed for refusal of alcohol service to visibly-intoxicated patrons, 
using pseudo-intoxicated patron (PiP) protocol (see 3C.14; primary outcome) at baseline (Months 15-17) (10 premises 
per state will be selected to replace any closed businesses). Using a custom-written program, project biostatistician will 
randomly assign n=80 premises to WayToServe training plus WayToServe Plus after baseline in Month 18, stratified by 
state and location. Remaining premises (n=80) will receive WayToServe training only. All premises will be recruited to 
have all servers complete WayToServe training. Servers in intervention group will be accepted into a Facebook private 
group after WayToServe training to receive in-service component with ~3 posts per week for 12 months (Months 18-29; 
n~160 posts total). In a prior trial, a social media campaign at this frequency over a year was effective (retention= 86%; 
see Box 4). In Months 30-32, all premises will be posttested with PiP assessments (blind to condition). Analyses will test 
the hypothesis:  
H: Compared to premises in WayToServe RBS training only group, premises assigned to receive WayToServe RBS 
training plus WayToServe Plus will have higher rates of refusing PiP at posttest. 
In our prior research, uptake of training in alcohol premises affected refusal rates,145 so we will test the research 
question of whether uptake of the WayToServe training and of WayToServe Plus improves PiP refusal rates. PiP 
assessment teams will also record characteristics of alcohol premises to be analyzed as effect modifiers of WayToServe 
Plus. The design will blind premises and alcohol servers to all PiP assessments, protecting against compensatory 
increases in refusal rates. Project staff, except Community Manager and Project Coordinator, also will be blind to 
experimental condition.  
3C.12. Implementation of WayToServe RBS Training  
Following baseline, all 180 premises will be contacted and recruited to the trial (premises compensation= $250 for 
intervention group and $150 for control group ). Managers will provide a list of all servers, who will be registered by 
project staff on WayToServe training and provided with a user ID that will track use and link it with premises. See Box 1 
for description of WayToServe RBS training. Servers 
will complete consent form and profile at their first 
login. They will be asked to complete the training 
within 4 weeks from registration; email and text 
reminders will be sent to those who do not login 
within 1 week (compensation= $35 from research 
team and new server training certificate from 
NMABC or WSLCB free of charge). WayToServe will 
remain available to premises throughout the trial; 
managers will report new hires so project staff can 
register them and they can complete the training. 
Drs. Buller and Woodall have successful protocols 
for recruiting premises: Over 400 onsite and off-site 
licensed premises were enrolled in the 3 trials 
evaluating WayToServe (English) and WayToServe 
Español (Spanish) training (see Box 1). After 
baseline assessment, we will call the premises 
management and arrange a meeting, where we will describe participation and obtain agreement for all servers and 
managers to participate in the study. We will appeal to the value-added benefits of server training to reduce violations, 
citations, and lawsuits. Based on our previous evaluations of WayToServe, we expect 80% of premises to agree to use 
WayToServe training (n=128 premises) and 80% of servers to complete the training (n=1024, assuming 10 servers per 
premises). Premises that decline will be recorded and scheduled for post assessments in intention-to-treat approach.  
3C.13. Implementation of WayToServe Plus In-service Professional Development Component 
The project biostatistician will randomize half of the premises to join WayToServe Plus following the baseline survey. 
Intervention-group trainees will be instructed to “friend” the Community Manager for the WayToServe Facebook page 
at the end of WayToServe training (compensation=$50). In-service component will be implemented through a Facebook 
private group that has the look and feel of the WayToServe Facebook page (see Box 3). Group members can comment 
and react to posts, but not share them on their own feed, so other Facebook friends cannot see private group or posts. 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
Based on current social media marketing advice for new Facebook algorithms, the Community Manager will post 
WayToServe Plus messages ~3 times per week (and post messages from WayToServe Facebook page the other 4 days) to 
achieve effective reach,141,146-148 scheduling posts in advance on Buffer software.149 Orientation to private groups will be 
self-explanatory. Alcohol servers in our preliminary survey (see Box 2) said they would check in-service component 
weekly, but not likely daily. In our past project (see Box 4), participants were adept at using Facebook private group and 
most commonly viewed posts at 8-9 am and 7-8 pm ET. The Community Manager will stress respect for others as a 
ground rule and monitor all comments and correct inappropriate/unfavorable/ bullying comments or misinformation, 
using a moderator guide that provides ways to respond while maintaining credibility (no bullying was observed in our 
past studies). If an alcohol server leaves the group, they will be contacted to see why; in our past project, 121 (13.7%) 
participants left groups (see Box 4). In the comparison group, servers will “follow” the WayToServe Facebook page 
where daily posts are focused on marketing WayToServe training (see Box 3).  
3C.14. Primary Outcome: Pseudo-patron Assessment of Alcohol Service to Intoxicated Patrons 
Primary outcome measure, refusal rate of alcohol service to apparently-intoxicated patrons, will be obtained via a 
pseudo-intoxicated patron (PiP) assessment. Confederate buyers will feign intoxication and record if servers agree to 
serve them an alcoholic beverage. PiP protocol presents a server with the most overt situation in which alcohol service 
should be refused, (i.e., when a patron is showing clear signs of intoxication), models behavior of patrons most at risk, 
and is relatively low cost. Other measures have drawbacks (e.g., unpredictable prevalence of intoxicated customers for 
onsite observation and biases in self-reports). PiP protocol has been used in thousands of alcohol premises;117 we have 
used it in over 400 premises. We will train ethnically-diverse male and female legal-age individuals (≥21) to conduct PiP 
assessments, blind to study purpose and treatment, chosen for prior acting experience and supervised by field 
supervisors. They will complete training from our past projects.150-152 Signs of intoxications trained (i.e., fumbling with 
keys/ cash, swaying, slurred speech, and stumbling) indicate high level of alcohol intoxication,153 provide a clear, 
unambiguous choice whether to serve, and are signs users learn to recognize in WayToServe. PiPs’ intoxication 
performance during training will be rated by project staff. PiPs will visit two premises not in sample, perform 
assessment, review with supervisor, and be re-trained, if needed. They will visit other premises, until they are 
comfortable with protocol, agree on 95% of characteristics, and observers report buyers display intoxication cues. 
Procedures for each visit are:  
1. Observer enters premises first, sits where they can observe the buyer, orders a non-alcoholic beverage, and records 
descriptive information on premises. 
2. Buyer enters premises exhibiting cues of intoxication and attempt to purchase an inexpensive beer.  
3. Observer notes buyer’s intoxication cues, buyer-server interaction, and if buyer is served an alcoholic beverage.   
4. After buyer is served or refused, buyer leaves the premises. If served, buyer pays for beverage and then leaves served 
drink untouched, excuses themselves for the restroom, and exits the premises.  
5. Observer waits 1-2 minutes after the buyer exits and then leaves the premises. 
After exiting, buyer and observer will complete buyer and observer data forms (see Appendix A), recording if server 
served an alcoholic beverage to buyer (yes v. no). Supervisors will check all forms. The observer serves as a quality 
control assessor of the PiP buyer’s presentation of feigned intoxication by recording if the buyer displayed signs of 
intoxication trained. Assessment will involve 2 PiP buyer visits per premises (separated by 6 weeks) in each assessment 
round (baseline, posttest), one with male buyer and one with female buyer. Gender has affected alcohol sales,154,155 so 
we will balance PiP buyers on gender and test gender pairs. Characteristics of buyer will be included as covariates to 
control variation introduced by multiple buyers (n~50). Descriptive data on premises (type – bar or restaurant; number 
of servers and patrons; environment [e.g., lighting, layout]) and alcohol server who interacts with PiP buyer (gender, 
age, and ethnicity; comments made about serving to buyer [e.g., not providing more than one drink or risk of DWI] and 
any other attempts not to serve requested alcoholic beverage) will be recorded in buyer/observer forms (buyers and 
observers had high agreement in our prior work; average Cohen’s kappa=0.85).150-152 These descriptive data will be 
tested as effect moderators.  
3C.15. Process Measures of WayToServe Training and WayToServe Plus Uptake  
Uptake of WayToServe training and WayToServe Plus will be recorded. For WayToServe training, number and 
proportion of servers registered and completing it at each premises and quiz/exam scores will be recorded. Servers will 
complete a Profile with name, contact information, premises, and role (owner, manager, server).156 For WayToServe 
Plus, servers’ reactions (e.g., like, love, wow, angry, and sad) and comments to all posts will be extracted in identified 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
format using the Grytics app157 (permission given in consent form; no data from their Facebook profile or personal 
newsfeed extracted). Two trained research assistants will code reactions/ comments as pro-, neutral, or anti-.158 
Facebook no longer displays views in private groups larger than n=250. After posttesting, intervention premises 
owners/managers (n=20; 10 per state) will be selected at random for interviews about WayToServe Plus, e.g., 
compatibility with premises RBS policies/practices, helpful features, server engagement, suggested improvements, and 
problems/barriers (compensation=$75).  
3C.16. Data Management 
Databases will be linked by a standard database structure and checked by sophisticated data editing/quality assurance 
methods. A random sample of 10% of manually-entered forms will be double entered; if error rate exceeds 1%, all forms 
will be double entered. Databases for PiP buyer/observer forms and process measures will be linked by a standard 
database structure. SAS 9.3 will be used in analyses. If missing data occur, analyses will be done a) on data collected 
(case-wise deletion), b) by intention-to-treat, and c) by imputation methods for missing at random data (if differential 
missingness is observed, by patterned mixture imputation).  
3C.17. Statistical Analysis Plan 
Dr. Cutter, the Project Biostatistician, will lead the analysis, working with Drs. Buller and Woodall and Ms. Liu, KB’s 
Biostatistical Manager (who has worked with Dr. Cutter for over 10 years).  
 
3C.17.1. Descriptive and Exploratory Analyses: Descriptive analyses (e.g., frequencies, means, SD) on refusals and 
premises/server characteristics will be calculated. Bivariate analyses (e.g., t-tests, χ2) will compare premises by 
treatment at baseline to identify differences, mindful that intervention effects are assessed in both groups. Attributes 
showing treatment differences exceeding 0.25 SD will be treated as covariates in sensitivity analyses.159 For primary 
analysis, there are 90 premises from intervention group (WayToServe training plus WayToServe Plus) and 90 premises 
from comparison group (WayToServe training only) (total N=180). 
3C.17.2. Test of Hypothesis: PiP Refusal Rates: Primary analyses will be done using intention-to-treat. Each 
premises will have two attempts to refuse to serve alcohol to PiP buyers; thus, primary outcome is refusal versus not 
refusing at posttest (i.e., 0 or 1 v. 2 refusals), assuming that 2 refusals is 
clear indication intervention had the desired outcome. The hypothesis 
(see 3C.11) will be tested by contrasting intervention with comparison 
premises. While each premises will have 2 before and 2 after PiP 
assessments, they will not be the same actors, but responses are 
clustered within premises. To simplify analysis, we will treat the 
premises as the unit of analysis, each scoring 0, 1, or 2 at baseline and 
posttest. Within intervention group, improvement in refusals is assessed 
using McNemar’s test, a χ2 test with 1 df computed as (A-B)2/(A+B), 
where A (green cells; Figure 8) and B (red cells) are off diagonal discordant pairs (A= N12+ N13+ N23; improved) and (B= 
N21+ N31+ N32; regressed). Both treatments should improve refusals. We will compare treatment groups using 
proportions improved (A/N) from baseline. Let us illustrate this analysis with data from our prior study on WayToServe 
training.11 In control group (n=77 onsite premises), A=(17+9+7)=33 and B=(9+2+5)=16, χ2=(33-16)2/(33+16)=5.9, p=0.015. 
In intervention group (n=71 premises), A=(15+14+12)=41 and B=(4+1+3)=8, χ2=(41-8)2/(41+8)=22.2, p< 0.001. Comparing 
the groups, we have 33/77=42.9% (control) and 41/71=57.7% (intervention), a significant improvement in training, 
p=0.049. We will use a 1-tailed p-value because 1) we have no evidence that RBS intervention is harmful and 2) the 
question is pragmatic: can we improve WayToServe training? Failure to reject the null hypothesis H0: Pwts>Pwts+ v. H0: 
Pwts<Pwts+ is in itself the decision not to implement WayToServe Plus.  
Additional analyses will enable adjustment for covariates using a repeated measures model with PROC GENMOD. 
Risk difference can be estimated using Margins or NLMeans macro, or by modeling the event probability directly. We 
will use hierarchical modeling, because observations (PiP buyer attempts) are nested within premises (Level II).160 Level I 
outcome measures are binary (e.g., refused v. not refused), so a logit link function will be used. The Level I model will 
include measurement point (baseline, posttest) as a fixed effect and an error term as a random effect. Level II will 
include treatment condition and premises characteristics as covariates. Sensitivity analyses will test covariates in 3C.14 
to ensure that results are not due imbalances.  
 
3C.17.3. Exploratory Research Questions: Effect of WayToServe training and WayToServe Plus in-service component 
uptake (see process measures in 3C.15) will be tested as a dose effect on PiP refusal rate. Uptake of training will be 
assessed in both treatment groups; uptake of the in-service component will only be available in the intervention group. 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
Dose-responses will be test as a simple covariate in the outcome models Receiver operator curve analysis161 will detect if 
certain uptake is clearly effective rather than just incremental. Potential effect moderators collected on alcohol servers, 
PiP buyers, and premises (see 3C.14), and state, will be tested within a regression framework using interaction terms of 
potential moderators with experimental condition, adjusted for multiple comparisons using a bonferroni correction. The 
differences associated with these moderators will need to be large to be detected. As ancillary tests, a 2-tailed p-value of 
0.05 will be used. 
3C.17.4. Statistical Power Analysis: For comparison of WayToServe training plus WayToServe Plus in-service 
component to WayToServe training only with the analysis approach in 3C.17.2, the final sample has n=90 premises in 
each group (n=180total), with two assessments rounds (baseline and posttest). The primary analysis will be to estimate 
the reduction (difference) in proportion of alcohol service (i.e., refusals: 0, 1, or 2) that occurred in the 2 PiPs per round 
and its 95% CI. Difference is expected to be a function of both training (both treatment groups) and in-service 
component (intervention group only). Our prior onsite trial showed an estimated 57.7% improvement for WayToServe 
training (see illustration of our approach to primary outcome analysis in 3C.17.2), but to be conservative we assume 50% 
to 55% (+10-15% in refusals). We expect that WayToServe Plus in-service component will increase refusal rates by 
another 20% to 70% to 75%. In the minimum difference, we will be comparing 75% in WayToServe Plus group to 55% in 
WayToServe training only group. A sample size of 70 per group provides 80% power with a 1-sided Type I error of 5%. To 
provide a buffer on the necessary improvement as not all premises will use WayToServe training and intervention 
servers will not view all WayToServe Plus posts, we plan to have 80 premises per group.   
Milestones and Deliverables: Milestone 7: Implementation and Randomized Trial Evaluation. Deliverable 10: Final 
Report on Evaluation of WayToServe Plus In-service Professional Development Component.  
3C.18. Strengths, Weaknesses, and Overall Impact of Proposed Research 
There are number of strengths. WayToServe Plus is based on effective RBS models. Randomized trial design is rigorous 
with adequate statistical power. PiP protocols provide objective measures. We avoided potential weaknesses. Blinding 
will reduce bias by not revealing the PiP assessments to owners, managers or servers; IRB-approved debriefing will be 
used. WayToServe training, social media development and delivery methods, and assessment protocols are already 
developed. The research will have very high impact. Intervention targets a frequent, highest-risk situation, alcohol 
intoxication, that if alcohol service continues produces dangerous impaired driving. In-service component can be 
immediately deployed with WayToServe training.  
 
 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
Eligibility Criteria 
Licensed Alcohol Premises: 
Inclusion Criteria: 
 
Located in California, New Mexico and Washington State 
 
Hold an active alcohol license from New Mexico Alcoholic Beverage Control or Washington State Liquor and 
Cannabis Board 
 
A bar or restaurant with full service bar 
Exclusion Criteria: 
 
Participated in our prior trials on WayToServe® RBS training  
 
Have an inactive, suspended or revoked alcohol license  
 
Alcohol Servers: 
Inclusion Criteria: 
 
19 years of age or older 
 
Own/manage a licensed premises or be an alcohol server (waitress, waiter or bartender) 
 
Have completed the WayToServe responsible beverage service training (Phase I only) 
 
Be proficient in English 
 
Consent to participate 
Exclusion Criteria: 
 
Have a family/household member already participating in project 
 
Younger than 19 years old (18 and younger cannot legally serve alcohol in either state) 
 
Age Limits 
Minimum Age: 19  
No maximum age 
 
Inclusion of Individuals Across the Lifespan: 
 
Children will not be included in the proposed research because state laws prohibit individuals under age 19 from serving 
alcohol in licensed premises in California, New Mexico and Washington State. Adults aged 18 will be excluded because they 
cannot legally sell alcoholic beverages in California, New Mexico and Washington State. There is no maximum age limit for 
participants. Age of all participants will be collected when enrolled in the research. 
 
Inclusion of Women and Minorities: 
We will recruit owners/managers and alcohol servers in on-site alcohol sales premises in California, New Mexico 
and Washington State at rates that reflect the gender and race/ethnic distribution in that workforce, based on our 
experience with WayToServe training and preliminary survey of WayToServe-trained alcohol servers. 
Combined the population of the two states: 59% of are female. Also, 80% are white, 4% Black/African American, 4% 
Asian, 0% Native Hawaiian or Other Pacific Islander, 2% American Indian/Alaska Native, and 10% more than one race. Of 
these, 20% are Hispanic. No exclusions will be made based on gender or race/ethnicity. The research team has 
experience in prior responsible beverage service (RBS) training research working with owners, managers, and servers in 
on- and off-site alcohol sales premises and we do not anticipate any difficulties in recruiting female and racial/ethnic 
minority participants.    
During the recruitment process, the Project Coordinator will screen for participants using IRB-approved protocols for 
screening based on gender, race, and ethnicity. If the Project Coordinator finds that we are over-recruiting a particular 
gender or racial or ethnic group, relative to our anticipated inclusion rate, we will place them on a waiting list until we 
have recruited an adequate number of targeted participants.  
 
Recruitment and Screening: 
Human Subjects oversight will be conducted by the Western Institutional Review Board (WIRB; DHHS IRB Reg. No. 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
IRB0000053; FWA No. 00003715). All subject materials (including recruitment materials, consent forms, and interview 
and discussion questions, protocols, measures, and data management procedures will be approved by the IRB prior to 
study implementation. Project staff will provide participants with IRB-approved consent forms that will describe the 
purpose of the project, risks and benefits, and selection criteria. Project staff will provide participants with copies of 
their signed IRB-approved consent forms that will describe the purpose of the project, risks and benefits, and selection 
criteria. Participants will be informed that they have a right to withdraw from the study at any time and there will be no 
penalty for non-participation.  
For the Phase I formative research, owners and managers of licensed alcohol premises will be recruited from 
publicly-available lists from New Mexico Alcoholic Beverage Control (NMABC) and Washington State Liquor and 
Cannabis Board (WSLCB). Participants for the focus groups (n= ~22), online survey (n= ~30), usability focus groups 
(n=20), and field pilot test (n=100) will be recruited from adults who complete the WayToServe responsible beverage 
service (RBS) training in New Mexico or Washington State. Adults will be recruited through email and written invitations, 
with telephone follow-up for owners/managers.  
For Phase II randomized trial, licensed alcohol premises will be recruited from publicly-available lists from NMABC 
and WSLCB. All of premises will be assessed with pseudo-intoxicated patron (PiP) assessment protocols without 
informing premises management (i.e., premises and staff are blind to assessment), because the PiP teams observe 
public behavior by alcohol servers (i.e., willingness to serve alcoholic beverage). This blinding should avoid the threat of 
compensatory behavior by premises and servers (i.e., increased refusal rates) due to study participation. Alcohol servers 
who encounter the PiP teams when they attempt to purchase an alcoholic beverage will be enrolled in the study. We 
will inform premises of the PiP assessments at the conclusion of the study, provide them with debriefing information, 
and permit them to withdraw. After baseline PiP assessents, Drs. Buller and Woodall will contact premises 
owners/managers by email and telephone and invite them to participate. They will schedule a meeting at each premises 
to explain the study and obtain a list of alcohol servers to be registered on the WayToServe training. Owners/managers 
will be provided unique codes for alcohol servers to access the WayToServe training online. This code will track each 
server’s use of the training and associate it with the premises. All alcohol servers at a participating premises will be 
invited to complete the WayToServe training. They will enter the training using a user code and complete a consent form 
and profile. They will be sent reminders to login and complete the training. Alcohol premises will be randomized to 
experimental condition, using a customized program prepared by the project biostatistician following completion of the 
baseline assessment, and alcohol servers in intervention premises will “friend” the Project Coordinator to be joined to 
the WayToServe Plus Facebook private group when they complete the WayToServe training. Servers in premises 
randomized to the WayToServe training only comparison group will follow the WayToServe Facebook page. In Year 3, KB 
staff will perform the posttest PiP assessment of all the premises in both treatment groups (blind to condition) at 12-
months post-randomization. Finally, an owner/manager at 20 participating intervention premises will be recruited to 
complete a semi-structured interview on the WayToServe Plus, as part of process measures. 
 
Retention and Adherence: 
For the formative focus groups, online survey, semi-structured interviews, and usability focus groups in Phase I, the 
length of time that participants are expected to remain in this study is approximately two hours. The formative field pilot 
test participants in Phase I will participate in the Facebook private groups, Qualtrics online survey, and semi-structured 
interviews (n=10 participants selected at random) over 4 weeks. To promote participation and retention, Phase I 
participants will be compensated for their time participating (focus groups=$75, online survey=$25, semi-structured 
interviews=$75, usability focus groups=$75, field pilot test=$100, field pilot test semi-structured interviews=$50).  
For the randomized trial in Phase II, the length of time that participants are expected to remain in this study is up to 
17 months. Participant retention and compliance with study procedures are paramount concerns and we will implement 
retention protocols from our prior trials on WayToServe RBS training and social media interventions. We will make 
expectations of the project activities very clear during this telephone and email contact. Locator information will be 
collected from owners, managers, and servers upon enrollment, including their physical address, email address, home, work, 
and mobile telephone numbers, and the addresses and phone numbers of a person who can always reach them. KB project 
staff will provide unique codes for alcohol servers to access the WayToServe training online, track each server’s use of 
the training, and associate it with the premises. Participants can withdraw at any time, so steps will be used to minimize 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
dropouts to ensure high internal validity. If a participant drops out of the private Facebook group, project staff will 
contact them to determine if they did so intentionally and if so, why. Participant progression will be tracked. As 
compensation, premises will receive $250 for intervention group and $150 for control group, servers who complete the 
WayToServe training will receive an alcohol server training certificate registered with NMABC or WSLCB and $35, and 
servers who join the WayToServe Plus Facebook private group will receive $50. WayToServe training will be available to 
the premises throughout the trial. We will select an additional 10 licensed premises as replacements for any that are 
discovered to be out-of-business at the baseline assessment. From prior experience in our WayToServe training and 
social media intervention trials, we are confident that we can obtain, engage, and retain a sample of high quality and 
generalizability. 
 
PROTECTION OF HUMAN SUBJECTS 
Risks to Human Subjects 
Human Subjects Involvement, Characteristics, and Design:   
The subject population for this study will be owners/managers and alcohol servers from premises licensed to sell 
alcohol by the drink (e.g., bars and restaurants) in California, New Mexico and Washington State. 
 
Study Procedures:  
Formative Semi-Structured Interviews (n=10 alcohol premise owners/managers): Participants will take part in one-
hour semi-structured interviews performed by Drs. Buller and Woodall, using KB’s video conferencing software, Zoom. 
Before the discussions, owners/managers will be sent the URL for the WayToServe online RBS training and instructed to 
review it. Owners/managers will discuss RBS policies at the premises, their support for RBS methods, the perceived 
importance of RBS, quality of RBS methods implemented by their servers, need for ongoing training and support for RBS 
among their servers, suggested content for in-service professional development for servers, and willingness to pay for 
WayToServe Plus.  Five participants will be from California, New Mexico and five from Washington State. Participants 
will be compensated $75 for their time. 
Formative Focus Group (n=~22 alcohol servers): Focus groups will be led by the Project Coordinator trained in focus 
group methods, using KB’s video conferencing software, Zoom. Servers will be recruited at random from the roster of 
WayToServe trainees in the past year. Participants will describe content and features, servers’ desire for in-service 
professional development in RBS methods, barriers to implementing RBS methods and means of overcoming them, 
potential utility of in-service component for premises and servers, and willingness to pay for WayToServe Plus. They will 
discuss the support they receive from premise owners/managers and other alcohol servers for RBS and its adequacy and 
their experience with RBS practices, especially after receiving state-mandated RBS training. They will be asked if they 
have ever refused a sale, how confident they were in doing so, and the techniques used to successfully or unsuccessfully 
refuse a sale. Servers will be asked if they would be willing to share RBS experiences or provide feedback on RBS actions 
to other servers on the WayToServe Facebook page. Approximately 11 participants will be from New Mexico and 
approximately 11 from Washington State. Participants will be compensated $75 for their time. 
Formative Online Survey with Alcohol Servers (n=~30 alcohol servers): This survey will be conducted through KB’s 
Qualtrics online survey software. Servers will be recruited at random from the roster of WayToServe trainees in the past 
year. Participants will describe content and features, servers’ desire for in-service professional development in RBS 
methods, barriers to implementing RBS methods and means of overcoming them, potential utility of in-service 
component for premises and servers, and willingness to pay for WayToServe Plus. They will discuss the support they 
receive from premise owners/managers and other alcohol servers for RBS and its adequacy and their experience with 
RBS practices, especially after receiving state-mandated RBS training. They will be asked if they have ever refused a sale, 
how confident they were in doing so, and the techniques used to successfully or unsuccessfully refuse a sale. Servers will 
be asked if they would be willing to share RBS experiences or provide feedback on RBS actions to other servers on the 
WayToServe Facebook page. Participants will be compensated $25 for completing the survey. 
 
Usability Focus Group (n=20 alcohol servers): Twenty alcohol servers will be recruited at random from the roster of 
WayToServe trainees (n=10 in New Mexico and 10 in Washington State) to participate in virtual focus group discussions 
on the acceptability, feasibility, and usability of the prototype WayToServe Plus led by the Project Coordinator and using 
KB’s video conferencing software, Zoom. Participants will be presented with a scenario describing the WayToServe Plus 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
and review the library of Facebook posts for this component. For each post, servers will identify the main message (to 
test comprehensibility), indicate if the content is appropriate and acceptable for alcohol servers, their premises, and 
their experience implementing RBS methods, report if the information will be useful to them and other alcohol servers, 
say whether they are likely to view, comment, or share the post, or contribute their own content on RBS methods to the 
Facebook page, and note what would keep them from engaging with posts (i.e., barriers to use). Participants will be 
compensated $75 for their time. Investigators and KB production staff will revise the WayToServe Plus based on this 
feedback and before field testing. 
 
Field Test of Prototype (n=100 alcohol servers): Participants will be recruited when they complete the WayToServe 
training. Participants (n=50 New Mexico; n=50 Washington State) will be randomized to WayToServe Plus or untreated 
control group. Servers in the WayToServe Plus group will be instructed to “follow” a WayToServe Plus Facebook private 
group, administered by the Project Coordinator (alcohol servers will be added by “friending” the Project Coordinator). It 
will have the look of the WayToServe Facebook page but the messages posted in this group will be viewable only to 
group members (i.e., study participants). The Project Coordinator will post three WayToServe Plus posts per week for 4 
weeks to the private group (n=12 posts), along with normal posts by WEDGE’s marketing staff. Alcohol servers’ 
engagement with WayToServe Plus will be assessed by recording number of times a post is viewed and commented on, 
and the number of posts made by alcohol servers (i.e., user-generated content such as personal experiences with 
refusing sales to intoxicated patrons or entry by minors). After 1 week, servers will complete a post-survey via a link to 
KB’s Qualtrics online survey software. It will assess acceptability/appropriateness of the in-service professional 
development component for alcohol servers, their premises, and their experience implementing RBS methods, the 
usability of the in-service component on the validated System Usability Scale (SUS), utility of the in-service posts to 
them, and perceived self-efficacy (confidence in practicing RBS) and response-efficacy (effectiveness of RBS methods for 
keeping community safe) for RBS methods. Servers will be asked how willing they would be to use the WayToServe Plus 
in the future and how they would improve the component. It will assess acceptability/appropriateness of the in-service 
professional development component for alcohol servers, their premises, and their experience implementing RBS 
methods, the usability of the in-service component on the validated SUS, utility of the in-service posts to them, and 
perceived self-efficacy (confidence in practicing RBS) and response-efficacy (effectiveness of RBS methods for keeping 
community safe) for RBS methods. Servers will be asked how willing they would be to use WayToServe Plus in the future. 
All participants will be asked how they would improve the component and make it more engaging, in open-ended 
questions. Participants will be compensated $100. 
 
Ten alcohol servers in the field test sample (n=5 New Mexico; 5 Washington State) will be randomly selected for a 
semi-structured interview on reactions to WayToServe Plus prototype. The Project Coordinator, using video 
conferencing, will probe for in-depth input on whether the in-service component was appropriate, acceptable, and 
useful for themselves, their premises, and their RBS methods, what made them engage with the posts or not, and how 
the in-service component could be improved and made more engaging. Participants will be compensated $50. 
 
Randomized Trial (n= 180 state-licensed onsite alcohol premises): The WayToServe Plus in-service component’s 
ability to increase refusal of service to intoxicated patrons will be tested in a randomized trial. A two-group randomized 
field trial (WayToServe training only [comparison control] v. WayToServe training plus WayToServe Plus in-service 
component [intervention]) with 2 assessment rounds (Baseline and Posttest [12-months post-randomization]) will yield 
a 2 (treatment) x 2 (assessment time) factorial design, with 4 assessments per alcohol premises (2 per round). Using a 
custom-written program, project biostatistician will randomly assign n=80 premises to WayToServe training plus 
WayToServe Plus after baseline in Month 18, stratified by state and location. Remaining premises (n=80) will receive 
WayToServe training only. All premises will be recruited to have all servers complete WayToServe training. Servers in 
intervention group will be accepted into a Facebook private group after WayToServe training to receive in-service 
component with ~3 posts per week for 12 months. In Months 30-32, all premises will be posttested with PiP 
assessments (blind to condition).  
 
Following baseline, all 180 premises will be contacted and recruited to the trial (premises compensation= $250 for 
intervention group and $150 for control group ). Managers will provide a list of all servers, who will be registered by 
project staff on WayToServe training and provided with a user ID that will track use and link it with premises. Servers will 
complete consent form and profile at their first login. They will be asked to complete the training within 4 weeks from 
registration; email and text reminders will be sent to those who do not login within 1 week (compensation= $35 from 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
research team and new server training certificate from NMABC or WSLCB free of charge). WayToServe will remain 
available to premises throughout the trial; managers will report new hires so project staff can register them and they 
can complete the training.  
 
The project biostatistician will randomize half of the premises to join WayToServe Plus after completing the 
WayToServe training. Intervention-group trainees will be instructed to “friend” the Community Manager for the 
WayToServe Facebook page at the end of WayToServe training (compensation=$50). In-service component will be 
implemented through a Facebook private group with the look and feel of the WayToServe Facebook page. Group 
members can comment and react to posts, but not share them on their own feed, so other Facebook friends cannot see 
private group or posts. Based on current social media marketing advice for the new Facebook algorithms, the 
Community Manager will post ~3 times per week to achieve effective reach (another 4 posts per week will be posted 
from the WayToServe Facebook page),256,295-297 scheduling posts in advance on Buffer software298 so they appear on a 
reliable schedule. Orientation to private groups will be self-explanatory.  
 
Study Materials:  
 
Data from interviews, focus groups, online surveys, and the field pilot test will be collected in the form of 
handwritten notes, video and audio recordings, computer files, transcriptions, and surveys. Transcripts of the recorded 
focus group and interview discussions will be analyzed via Atlas-ti© content and thematic analysis software to identify 
consistent and recurrent content and themes in response to discussions. Data gathered from the formative online 
survey with alcohol servers will be collected using KB’s Qualtrics online survey software, and analyzed using various 
quantitative methods to confirm and/or learn more about the themes identified in the aforementioned focus group and 
interview discussions. Data gathered from the usability focus group will be summarized in notes by research staff to 
identify usability problems. Data gathered from the randomized trial will be collected via pseudo-intoxicated patron 
(PiP) assessments. Registration on and progress through the WayToServe RBS training will be tracked by the training’s 
backend SQL database, along with scores on quizzes/final exam. Users’ engagement with the WayToServe Plus will be 
assessed by recording number of reactions and comments to a post, and posts made by alcohol servers with user-
generated content. All data will be obtained specifically for research purposes. Data will be kept confidential and 
anonymous. Access to all data will be limited to the Principal Investigator (PI), Co-Investigator, specific project staff, and 
consultants. All data will be kept in locked files and secure computer servers. 
 
Potential Risks:   
     The potential risks to participants are minimal, mainly social or psychological. Focus group, interview, and survey 
participants may be hesitant or uncomfortable responding to questions about alcohol service in licensed establishments 
and methods to prevent sales to intoxicated patrons and minors. Participants in the usability testing and field pilot test 
may not feel comfortable responding to questions about their opinions on appeal, ease of understanding, usability of, 
and engagement with the WayToServe Plus in-service professional development component. Owners/managers/clerks 
in the randomized trial will participate anonymously and passively in the PiP assessments and the data from these 
measures on individual stores and persons will be kept confidential and not shared with state regulators, other 
premises, or anyone outside the KB research team. Data on specific servers will not be reported to the premises. In any 
group-based intervention, participants breaking confidentiality is a possibility and risks revealing private communication 
by and information about other participants to individuals outside the study. This is considered a minimal risk. We will 
add it to the consent forms and warn participants against making such disclosures when joining the trial and the 
Facebook private group. There is a risk that some randomized trial participants may experience hostile communication 
or “cyberbullying.” Participants may choose to not answer questions, as required the by WIRB. Any public release of the 
data will be in aggregate form, summarizing across groups of premises and individuals. These methods reduce the risk to 
trial participants. All individual responses collected in this project will not be shared with anyone except KB project staff. 
Participants will be allowed to withdraw at any time if they are uncomfortable with aspects of the research. No 
additional data will be collected from participants who decide to withdraw; however, the information collected prior to 
withdrawal will remain in the study.  
 
ADEQUACY OF PROTECTION AGAINST RISKS  

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
Informed Consent:  
Human Subjects oversight will be conducted by the WIRB (DHHS IRB Reg. No. IRB0000053; FWA No. 00003715). All 
subject materials (including recruitment materials, consent forms, survey questions, interview questions, protocols and 
measures) and data management procedures will be approved by the IRB prior to study implementation. 
Since the Phase I formative research activities will be conducted using KB’s video conferencing software, Zoom, 
participants will be presented with an online IRB-approved consent statement.  Participants will then click on a button 
that indicates whether they consent to participate. Similarly, for the field usability test, field pilot test, and the 
randomized trial, participants will be presented with a written consent statement and participants will click on buttons 
that indicate whether they consent to participate. The consent statements will describe the purpose of the project, risks 
and benefits, and selection criteria. Participants will be given an opportunity to ask questions and have them answered. 
Participants will have the option to print or save the consent statement. 
All participants will be informed that their consent to participate is completely voluntary and that they will be able 
to skip over any questions that make them feel uncomfortable. Responses to survey, interview, focus group, or field 
pilot test questions will not reasonably place participants at risk of criminal or civil liability or be damaging to the 
subjects' financial standing, employability, or reputation. Participants will be informed that thei responses will be 
confidential, they have a right to withdraw from the study at any time, and there will be no penalty for non-
participation.  
All data management, analysis and reporting activities will be conducted by KB staff. All notes and printed data 
collection forms will be stored in locked cabinets; all electronic data files will be stored on secure network servers 
behind computer firewalls, with routine backup. No clinical data will be captured. All data coding, data entry, analysis 
and reporting activities will be conducted by KB. All identifiers for participants will be stored in secure files and behind 
computer firewalls at KB. Participant names and study identification numbers will be recorded and stored separately 
from the research data files. Data forms will be labeled only with participant identification number. Results of analyses 
will be shared in aggregate form only. No additional data will be collected from participants who decide to withdraw; 
however, the information collected prior to withdrawal will remain in the study.  
 
Protection Against Risk: 
To minimize potential psychological risks, staff, facilitators, and interviewers will establish ground rules for discussion 
that includes an open, non-evaluative exchange of comments and ideas. All data collection forms and surveys will be 
IRB-approved and administered under the supervision of senior project staff. Participants will be told that they do not 
have to answer any questions that make them uncomfortable and can terminate their participation at any time they 
wish. Notes, surveys, video recordings, transcripts, and data will be confidential, accessible only by research personnel. 
Data forms and computer records will be labeled only with participant identification number. Participant names and 
study identification numbers will be recorded and stored separately from the research data files. Focus group 
participants will be asked to use first names only during focus groups.   
IRB-approved survey instruments and surveying procedures will be carried out under the supervision of project staff. 
Survey questions will be de-identified before being submitted for analysis and the Project Coordinator will remove any 
references to individual names or identifier data. Responses to survey questions would not reasonably place the 
subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing, employability, or 
reputation. The data collected on this project will be obtained with the "use of educational tests, survey questions and 
interview procedures."  
All data collection forms for the PiP assessment will be reviewed and approved by WIRB and used under the 
supervision of Investigators and the Project Coordinator. In our past trials testing WayToServe RBS training, the IRBs 
approved selecting and assessing alcohol premises with pseudo-patron protocols without informing premises 
management (i.e., premises were blind to assessment), ruling it was an assessment of public behavior. Thus, all alcohol 
premises and alcohol servers in the randomized trial will be blind to all PiP assessments at baseline and posttest. In our 
past trials, the IRBs required us to disclose the premises’ participation at the end of the study. We will inform premises 
at the conclusion of the study, provide them with debriefing information, and permit them to withdraw. To minimize 
potential psychological risk, we will make it very plain that data collected as part of the research study will not be 
available to law enforcement and the project will receive a Certificate of Confidentiality from NIH. 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
The prototype WayToServe Plus in-service professional development component and databases will be hosted on 
KB’s state-of-the-art web server farm. To minimize the risk to participant data privacy and protection through the 
Internet, KB programmers will monitor and maintain all programs and databases and use an established protocol for 
participant cyber safety and security. KB is HIPAA complaint and maintains vigorous technical safeguards which are built 
into KB’s IT system to protect information and to control access to it.   
To reduce risks to participants in the randomized trial, servers in intervention group will be invited to a private 
Facebook group. In private groups, only members can view posts and there is no option to “share” content onto one’s 
main profile page. This will keep content only accessible via the group. Only the group administrator can invite people to 
the group. At the outset of the study, alcohol servers will be asked to keep all discussion in the group 
confidential.  Servers will be informed as to the nature of the privacy settings (e.g., all participants can see their posts in 
the group). They will also be reminded to update their personal Facebook page privacy settings if they want to protect 
against other participants inspecting their personal pages.  As in face-to-face groups, participants will know each other’s 
names and be exposed to any information that group members wish to share with each other. In the same way we ask 
participants to keep all group discussions confidential for in-person groups, we will do so for Facebook discussions.   
Data will be hosted on KB’s state-of-the-art web server farm. To minimize the risk to participant data privacy and 
protection through the Internet, KB programmers will monitor and maintain all programs and databases and use an 
established protocol for participant cyber safety and security. KB is HIPAA complaint and maintains vigorous technical 
safeguards which are built into KB’s IT system to protect information and to control access to it. All participant 
information will be protected using 128-bit SSL encryption. All Internet information must travel through KB’s CISCO ASA 
5505 firewall using ASA software version 9.0(2) encrypted 3DES-AES. Traffic then moves to KB’s web server. This server 
is a Dell Power Edge T-310 with 32 GB of RAM, 1.5 TB available disk space on a RAID 5 redundant drive system using 
Windows Hyper-V Operating System. The virtual web server uses Windows 2008 IIS Web server software. User click 
stream data is collected using Webtrends and is saved to our SQL server. Enrollment data is transferred to our SQL 
server (Windows Server 2008) which uses MS SQL server 2008 R2 database software. This machine is also a Dell Power 
Edge T-310 with 32 GB of RAM, 1.5 TB available disk space on a RAID 5 redundant drive system using Windows Hyper-V 
OS and has no access to the Internet. When a participant is directed to take a survey, their computer is linked to a third 
server in our system which is also a Dell Power Edge T-310 with 32 GB of RAM, 1.5 TB available disk space on a RAID 5 
redundant drive system using Windows Hyper-V OS. This server runs Qualtrics survey software. All network traffic is sent 
from the CISCO firewall to a Syslog server using Manage Engine Firewall Analyzer 7 software. Login and network activity 
are required and monitored by Windows servers to an event viewer. Servers are kept in a locked room on site. All files 
are secured locally using MS NTFS. All traffic between servers and PCs on the LAN are digitally signed communications. 
Each local machine has a built-in operating system firewall. Data from the Qualtrics survey software are exported for 
data analysis by local machines which are also behind KB’s firewall. Access to this exported data by outside sources is 
done using File Transfer Protocol (FTP) protected by Secure Socket Layer (SSL). SQL databases are protected by the built-
in security of MS Active Directory. Data is backed up using a Quantum SuperLoader 3 tape library using LT04 tapes that 
contain 800 GB/1.6 TB of info (depending on compression used). The device holds 16 tapes. The backup software used 
by KB is Symantec Back-up Exec. 2012. Daily backups are incremental and tapes are recycled weekly. Weekly backups 
are done every Friday and are recycled every six weeks. Monthly backups are done on the last Friday of each month and 
are full backups of all data. These monthly tapes are stored in a secure offsite facility and are never recycled. 
 
POTENTIAL BENEFITS OF THE PROPOSED RESEARCH TO PARTICIPANTS AND OTHERS: 
The research literature shows that training and certification of community members in prevention activities improves 
professionalism and along with on-going in-service contact has boosted effectiveness of community-based 
interventions. Alcohol servers may benefit from ongoing in-service contact aimed at improving their professionalism by 
a) motivating them to implement the RBS skills in the face of common barriers such as pressure to sell, management 
disinterest or resistance, and customers’ attempts to continue being served, b) providing support for RBS actions from a 
“community” of alcohol servers, especially for servers who work in small or unsupportive premises, and c) preventing 
natural degradation of skills over time by having them self-enroll into booster training on RBS methods. Potential 
benefits for all participants will be the knowledge that one has helped in development of an important prevention 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
intervention that may be used on a national basis to reduce the number of alcohol-related problems, including driving 
while under the influence of alcohol. 
A description of this study will be available on http://www.ClinicalTrials.gov, as required by U.S. Law. This website will 
not include information that can identify participants. At most, the website will include a summary of the results. This 
website can be searched at any time. 
  
IMPORTANCE OF THE KNOWLEDGE TO BE GAINED: 
We are proposing the development of an innovative online RBS in-service professional development program that 
has the potential to increase the efficacy of existing evidence-based RBS training. The sale of alcohol to patrons who are 
either on the verge of intoxication or already are intoxicated, and the consequent instances of DWI and related crashes, 
injury, and death, remain a very serious and preventable public health issue, among the most serious in the United 
States. Among interventions that can be used to prevent DWI and its consequences, RBS training holds promise as we 
have demonstrated in our previous R01 funded research with alcohol servers in onsite (by the drink) and off-site 
(package) alcohol sales premises. Interventions that intervene at the point of alcohol consumption before a drinker 
becomes intoxicated and attempts to drive such as RBS training, rather than after they reach intoxication and attempt to 
drive (e.g., roadside checkpoints or ignition interlock devices), can be an important part of multi-component 
interventions that also include premises management training and environmental policies and are seen as positive by 
many alcohol servers who wish to provide alcohol in careful, responsible ways that keep their customers safe.  
 
Data and Safety Monitoring Plan: 
 
A data and safety monitoring plan will be implemented for this project, following well-established procedures. We 
will not set up an independent Data Safety and Monitoring Board because this study is very low risk, does not include 
use of therapeutics, and psychological risk is not anticipated. A total of 152 alcohol servers and 10 owners/managers of 
licensed alcohol premises will be enrolled in Phase I formative research. Also, 180 licensed alcohol premises in 
California, New Mexico and Washington State will be enrolled in the randomized trial and assessed for refusal of sales 
to pseudo-intoxicated buyers, using a PiP assessment protocol. Alcohol servers from participating premises (up to 
n=1600, 10 per premises) will complete the WayToServe RBS training and servers (n=800) assigned to the intervention 
group will use the WayToServe Plus in-service professional development component. Finally, an owner/manager at 20 
intervention premises will complete a semi-structured interview on the use of the WayToServe Plus in-service 
component.  
 
The progress of the research will be monitored monthly by the Multiple PIs, Co-Investigator, Project Coordinator, 
and Project Biostatistician. We will maintain strict confidentiality of participants in the project’s registration system, 
provide only ID numbers in data files, and ensure quality control using standard data management routines. A serious 
adverse event will be any problems that cause the project to be “banned” from a premises; critical events will be 
reviewed by the Investigators to ensure that project methods have not been intrusive or disruptive. Compliance with 
data collection protocols, including the PiP assessment protocol, will be monitored throughout data collection by the 
Investigators and Project Coordinator. The data management staff at KB will examine the quality of data input 
throughout the data collection period following standard quality control and assurance procedures (e.g., double entry of 
hard-copy coded data; detection and estimates of keystroke errors, identification of systematic mistakes in coding or 
responses; range checks). Analyses of the data will be performed by Dr. Cutter and Ms. Liu, at the conclusion of the 
baseline data collection periods to assess distributional assumptions and the success of randomization.  
Annual reports on data safety and monitoring will be submitted to WIRB. Dr. Buller, Contact Multiple PI and the 
Project Coordinator are responsible for reporting all adverse events to WIRB. Only Grade 1 events are expected on this 
trial. Thus, adverse events will be reported annually, as required by Department of Health and Human Services. This 
report will be reviewed by WIRB. Any action taken by WIRB or project investigators resulting in a temporary or 
permanent suspension of the trial will be communicated immediately by Dr. Buller, Multiple PI to the NIH program 
officer. 
 
 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
References 
 
1. 
Shults RA, Elder RW, Sleet DA, et al. Reviews of evidence regarding interventions to reduce alcohol-impaired 
driving. Am J Prev Med. 2001;21(4):66-88. 
2. 
National Highway Traffic Safety Administration. Traffic Safety Facts: Alochol-Impaired Driving. Washington, D.C.: 
U.S. Department of Transportation, National Highway Traffic Safety Administration;2019. DOT HS 812 864. 
3. 
National Center for Statistics and Analysis. 2017 Fatal Motor Vehicle Crashes: Overview. Washington, D.C.: U.S. 
Department of Transportation, National Highway Traffic Safety Administration; 2018. DOT HS 812 603. 
4. 
Chang I, Lapham SC, Barton KJ. Drinking environment and sociodemographic factors among DWI offenders. J 
Stud Alcohol. 1996;57(6):659-669. 
5. 
Gruenewald PJ, Stockwell T, Beel A, Dyskin EV. Beverage sales and drinking and driving: the role of on-premise 
drinking places. J Stud Alcohol. 1999;60(1):47-53. 
6. 
Stockwell T, Lang E, Rydon P. High risk drinking settings: the association of serving and promotional practices 
with harmful drinking. Addiction. 1993;88(11):1519-1526. 
7. 
Ker K, Chinnock P. Interventions in the alcohol server setting for preventing injuries (Review). Cochrane 
Database Syst Rev. 2008;16(3):CD005244. 
8. 
Foxcroft DR. Can Prevention Classification be Improved by Considering the Function of Prevention? Prev Sci. 
2014;15(6):818-822. doi: 10.1007/s11121-013-0435-1 
9. 
Lenk KM, Toomey TL, Nelson TF, Jones-Webb R, Erickson DJ. State and local law enforcement agency efforts to 
prevent sales to obviously intoxicated patrons. J Community Health. 2014;39(2):339-348. doi: 10.1007/s10900-
013-9767-9.PMC3943669 
10. 
Dresser J. Comparing statewide alcohol server training systems. Paper presented at: 15th International 
Conference on Alcohol, Drugs and Traffic Safety; September 22-26, 2000; Stockholm, Sweden.  
11. 
Woodall WG, Starling R, Saltz RF, Buller DB, Stanghetta P. Results of a randomized trial of web-based retail 
onsite responsible beverage service training: WayToServe. J Studies Alcohol  Drugs. 2018;79(5):672-
679.PMC6240009 
12. 
Ingram M, Reinschmidt KM, Schachter KA, et al. Establishing a professional profile of community health workers: 
results from a national study of roles, activities and training. J Community Health. 2012;37(2):529-537. doi: 
10.1007/s10900-011-9475-2 
13. 
Woodruff SI, Candelaria JI, Elder JP. Recruitment, training outcomes, retention, and performance of community 
health advisors in two tobacco control interventions for Latinos. J Community Health. 2010;35(2):124-134. doi: 
10.1007/s10900-009-9207-z 
14. 
Horwood C, Butler L, Barker P, et al. A continuous quality improvement intervention to improve the 
effectiveness of community health workers providing care to mothers and children: a cluster randomised 
controlled trial in South Africa. Hum Resour Health. 2017;15(1):39. doi: 10.1186/s12960-017-0210-
7.PMC5470211 
15. 
Blumenthal D, Kilo CM. A report card on continuous quality improvement. Milbank Q. 1998;76(4):625-648, 
511.PMC2751093 
16. 
Bureau of Labor Statistics, U.S. Department of Labor. Occupational employment statistics, occupational 
employment and wages, May 2019: 35-3031 waiters and waitresses. Occupational Employment Statistics. U.S. 
Bureau of Labor Statistics. Available at: https://www.bls.gov/oes/current/oes353031.htm#(1).  Updated July 6, 
2020. Accessed September 1, 2020. 
17. 
Data USA: bartenders. Data USA Web site. Available at: https://datausa.io/profile/soc/bartenders. Accessed 
September 1, 2020. 
18. 
Data USA: waiters & waitresses. Data USA Web site. Available at: https://datausa.io/profile/soc/waiters-
waitresses#employment. Accessed September 1, 2020. 
19. 
Veil SR, Buehner T, Palenchar MJ. A work-in-process literature review: incorporating social media in risk and 
crisis communication. J Contingencies Crisis Manage. 2011;19(2):110-122. doi: doi:10.1111/j.1468-
5973.2011.00639.x 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
20. 
Breland JY, Quintiliani LM, Schneider KL, May CN, Pagoto S. Social media as a tool to increase the impact of 
public health research. Am J Public Health. 2017;107(12):1890-1891. doi: 
10.2105/ajph.2017.304098.PMC5678392 
21. 
Sutton J. Health communication trolls and bots versus public health agencies’ trusted voices. Am J Public Health. 
2018;108(10):1281-1282. doi: 10.2105/ajph.2018.304661 
22. 
Internet/Broadband Fact Sheet. Available at: http://www.pewinternet.org/fact-sheet/internet-broadband/.  
Published June 12 2019.  Accessed August 28, 2020. 
23. 
Korda H, Itani Z. Harnessing social media for health promotion and behavior change. Health Promotion Practice. 
2011;14(1):15-23. 
24. 
Portnoy DB, Scott-Sheldon LAJ, Johnson BT, Carey MP. Computer-delivered interventions for health promotion 
and behavioral risk reduction: a meta-analysis of 75 randomized controlled trials, 1988-2007. Prev Med. 
2008;47(1):3-16. 
25. 
Social networking fact sheet. PewResearch Internet Project Web site. Available at: 
http://www.pewinternet.org/fact-sheets/social-networking-fact-sheet/. Published 2014. Accessed September 4, 
2020. 
26. 
Duggan M, Brenner J. The demographics of social media users - 2012. Pew Internet & American Life Project. 
Available at: http://pewinternet.org/Reports/2013/Social-media-users/Social-Networking-Site-Users/Demo-
portrait.aspx. Published 2013. 
27. 
; Pew Research Center. Mobile Fact Sheet. Available at: https://www.pewinternet.org/fact-sheet/mobile/.  
Published June 12 2019.  Accessed September 4, 2020. 
28. 
Smith A, Anderson M. Social Media Use in 2018. Available at: http://www.pewinternet.org/2018/03/01/social-
media-use-in-2018/.  Published March 1 2018.  Accessed September 4, 2020. 
29. 
Rogers EM. Diffusion of Innovations. 5th ed. New York, NY: Free Press; 2003. 
30. 
Lenz ER. Information seeking: a component of client decisions and health behavior. Advances in Nursing Science. 
1984;13(6):59-72. 
31. 
Pescosolido B. Beyond rational choice: the social dynamics of how people seek help. Amer J Sociol. 
1992;97(4):1096-1138. 
32. 
Reagan J, Collins J. Sources of health care information in two small communities. The Journalism Quarterly. 
1987;64(3):560-563. 
33. 
Hughes A. Using Social Media Platforms To Amplify Public Health Messages. An Examination Of Tenets And Best 
Practices For Communicating With Key Audiences. Available at: http://csic.georgetown.edu/wp-
content/uploads/2016/12/public-health.pdf.  Published November 2010.  Accessed September 4, 2020. 
34. 
Himelboim I, Xiao X, Lee DKL, Wang MY, Borah P. A social networks approach to understanding vaccine 
conversations on Twitter: network clusters, sentiment, and certainty in hpv social networks. Health Commun. 
2019:1-9. doi: 10.1080/10410236.2019.1573446 
35. 
Hoffman BL, Felter EM, Chu K-H, et al. 266. The emerging landscape of anti-vaccination sentiment on facebook. J 
Adolesc Health. 2019;64(2):S136. doi: 10.1016/j.jadohealth.2018.10.283 
36. 
Hoffman BL, Felter EM, Chu KH, et al. It's not all about autism: the emerging landscape of anti-vaccination 
sentiment on Facebook. Vaccine. 2019;37(16):2216-2223. doi: 10.1016/j.vaccine.2019.03.003 
37. 
Cramér H. Mathematical Methods of Statistics (PMS-9). Princeton University Press; 1999. 
38. 
Turner J. Towards a cognitive redefinition of the social group. In: Tajfel, H, ed. Social Identity and Intergroup 
Relations. New York, NY: Academic Press; 1982:15-40. 
39. 
Turner RH, Killian LM. Collective Behavior. 3rd ed. Englewood Cliffs, NJ: Prentice-Hall; 1992. 
40. 
Erickson BH. The relational basis of attitudes. In: Wellman, B, Berkowitz, SD, eds. Social Structures: A Network 
Approach. Cambridge, England: Cambridge University Press; 1988. 
41. 
Suls JM, Miller RL. Social Comparison Processes. New York, NY: Hemisphere; 1977. 
42. 
Festinger L. A theory of social comparison processes. Human Relations. 1954;7(2):117-140. 
43. 
National Highway Traffic Safety Administration. Drunk-driving. National Highway Traffic Safety Administration 
Web site. Available at: https://www.nhtsa.gov/risky-driving/drunk-driving. Accessed September 3, 2020. 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
44. 
National Institute on Alcohol Abuse and Alcoholism. Underage drinking. National Institutes of Health Website. 
Available at: http://www.niaaa.nih.gov/alcohol-health/special-populations-co-occurring-disorders/underage-
drinking. Accessed August 31, 2020. 
45. 
Miller TR, Levy DT, Spicer RS, Taylor DM. Societal costs of underage drinking. J Stud Alcohol. 2006;67(4):519-528. 
46. 
Impaired driving in the United States. National Highway Traffic Safety Administration. Available at: 
http://www.nhtsa.gov/people/injury/alcohol/impaired_driving_pg2/US.htm. Accessed August 31, 2020. 
47. 
Chain restaurants in the US number of businesses 2001–2026. IBISWorld Web site. Available at: 
https://www.ibisworld.com/industry-statistics/number-of-businesses/chain-restaurants-united-states/. 
Accessed September 3, 2020. 
48. 
Lock S. Sales of the leading chain restaurants in the U.S. 2019. Statista Web site. Available at: 
https://www.statista.com/statistics/298191/sales-of-the-leading-chain-restaurants-in-the-us/. Published 2020. 
Accessed September 3, 2020. 
49. 
Girl, 4, served alcoholic Mudslide at Chili's, mom says. CBS Chicago. 2011. 
http://chicago.cbslocal.com/2011/04/18/girl-4-served-alcoholic-mudslide-at-chilis/. Accessed August 31, 2020. 
50. 
Texas. Glazer's. Available at: http://www.glazers.com/locations/texas/Pages/default.aspx. August 31, 2020. 
51. 
Gilbert K. Which state has the most casinos? WorldAtlas. August 1, 2017. 
https://www.worldatlas.com/articles/which-state-has-the-most-casinos.html. Accessed September 3, 2020. 
52. 
Lock S. U.S. gambling market: total revenue 2004-2018. Statista Web site. Available at: 
https://www.statista.com/statistics/271583/casino-gaming-market-in-the-us/. Published 2020.  Updated May 
29, 2020. Accessed September 3, 2020. 
53. 
American Gaming Association, GamblingCompliance. State of the States 2019: the AGA Survey of the 
Commercial Casino Industry. Available at: https://www.americangaming.org/wp-
content/uploads/2019/06/AGA-2019-State-of-the-States_FINAL.pdf. 2019.  Accessed September 3, 2020. 
54. 
Casino alcohol policies. American Gaming Association Website. Available at: 
http://www.americangaming.org/industry-resources/research/fact-sheets/casino-alcohol-policies. Accessed 
August 31, 2020. 
55. 
Government affairs: state guilds. Brewers Association. Available at: 
https://www.brewersassociation.org/government-affairs/state-guilds/. Accessed September 3, 2020, 2020. 
56. 
About us. American Wine Society Web site. Available at: https://americanwinesociety.org/about/. Accessed 
September 2, 2020. 
57. 
State guilds. American Craft Spirits Association Web site. Available at: https://americancraftspirits.org/industry-
sponsors/state-guilds/. Accessed September 3, 2020. 
58. 
Total U.S. restaurant count at 660,755 in Spring 2018, a one percent drop from last year, reports NPD [press 
release]. Chicago, IL: The NPD Group, August 22.Available at:  
https://www.npd.com/wps/portal/npd/us/news/press-releases/2018/total-us-restaurant-count-at-660755-in-
spring-2018-a-one-percent-drop-from-last-year-reports-npd/. 
59. 
State craft beer sales & production statistics, 2019. Brewers Association Web site. Available at: 
https://www.brewersassociation.org/statistics-and-data/state-craft-beer-stats/. Accessed September 2, 2020. 
60. 
The year in beer: 2014 craft beer in review from the Brewer's Association [press release]. Brewer's Association, 
December 9.Available at:  http://www.brewersassociation.org/press-releases/year-beer-2014-craft-beer-
review-brewers-association/. Accessed August 31, 2020. 
61. 
Morse P. U.S. Hispanics and the beer industry. The Marketing Insider. May 14, 2018. 
https://www.mediapost.com/publications/article/319207/us-hispanics-and-the-beer-industry.html. Accessed 
September 2, 2020. 
62. 
State and county quickfacts: California. U.S. Census Bureau Website. Available at: 
http://quickfacts.census.gov/qfd/states/06000.html.  Updated March 31, 2015. Accessed August 31, 2020. 
63. 
Dev R. Patent protection of mobile [smartphone] applications - Android, Apple, Microsoft & Blackberry apps. 
Advocate Rahul Dev Web site. https://advocaterahuldev.com/patent-protection-of-mobile-smartphone-
applications-android-apple-microsoft-blackberry-apps-d8ced1309fe5. Accessed September 4, 2020.  

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
64. 
New Mexico Wine events: Las Cruces Wine Festival. New Mexico Wine Web site. Available at: 
https://nmwine.com/events/lc-wine-fest/. Accessed September 4, 2020. 
65. 
Purchase your tickets now! Brewmasters Craft Beer Festival Website. Available at: 
https://www.brewmastersbeerfest.com/p/purchase-your-tickets-now. Accessed August 31, 2020. 
66. 
Home page. Big Texas Beerfest Website. Available at: http://www.bigtexasbeerfest.com/. Accessed September 
2, 2020. 
67. 
Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J 
Usability Stud. 2009;4(3):114-123. 
68. 
National Safety Council. Motor vehicle: introduction. Injury Facts. National Safety Council's Injury Facts Web site. 
Available at: https://injuryfacts.nsc.org/motor-vehicle/overview/introduction/. Accessed August 31, 2020. 
69. 
Taylor B, Rehm J. The relationship between alcohol consumption and fatal motor vehicle injury: high risk at low 
alcohol levels. Alcohol Clin Exp Res. 2012;36(10):1827-1834. doi: 10.1111/j.1530-
0277.2012.01785.x.PMC3433627 
70. 
Taylor B, Irving HM, Kanteres F, et al. The more you drink, the harder you fall: a systematic review and meta-
analysis of how acute alcohol consumption and injury or collision risk increase together. Drug Alcohol Depend. 
2010;110(1-2):108-116. doi: 10.1016/j.drugalcdep.2010.02.011.PMC2887748 
71. 
Irwin C, Iudakhina E, Desbrow B, McCartney D. Effects of acute alcohol consumption on measures of simulated 
driving: A systematic review and meta-analysis. Accid Anal Prev. 2017;102:248-266. doi: 
10.1016/j.aap.2017.03.001 
72. 
Baldock MR, Lindsay VL. Examination of the role of the combination of alcohol and cannabis in South Australian 
road crashes. Traffic Inj Prev. 2015;16(5):443-449. doi: 10.1080/15389588.2014.969804 
73. 
Compton R. Marijuana-Impaired Driving - A Report to Congress. Washington, D.C.: National Highway Traffic 
Safety Adminstration;2017. DOT HS 812 440. 
74. 
Fell JC, McKnight AS, Auld-Owens A. Increasing Impaired-Driving Enforcement Visibility: Six Case Studies. 
Washington, DC: National Highway Traffic Safety Administration;2013. DOT HS 811 716. 
75. 
Biglan A, Flay BR, Embry DD, Sandler IN. The critical role of nurturing environments for promoting human well-
being. Am Psychol. 2012;67(4):257-271. doi: 10.1037/a0026796.3621015 
76. 
Toomey TL, Lenk KM, Erickson DJ, et al. Effects of a Hybrid Online and In-Person Training Program Designed to 
Reduce Alcohol Sales to Obviously Intoxicated Patrons. J Stud Alcohol Drugs. 2017;78(2):268-275.PMC5554107 
77. 
Graham K. Preventive interventions for on-premise drinking: a promising but underresearched area of 
prevention. ntemp Drug Probl. 2000;27(3):593-668. 
78. 
Buvik K, Rossow I. Server training at drinking establishments: a sisyphean task? A commentary on Toomey et al. 
(2017). J Stud Alcohol Drugs. 2017;78(2):276-277. 
79. 
Wallin E, Gripenberg J, Andreasson S. Overserving at licensed premises in Stockholm: effects of a community 
action program. J Stud Alcohol. 2005;66(6):806-814. 
80. 
Gripenberg J, Wallin E, Andreasson S. Effects of a community-based drug use prevention program targeting 
licensed premises. Subst Use Misuse. 2007;42(12-13):1883-1898. doi: 10.1080/10826080701532916 
81. 
Gripenberg Abdon J, Wallin E, Andreasson S. Long-term effects of a community-based intervention: 5-year 
follow-up of 'Clubs against Drugs'. Addiction. 2011;106(11):1997-2004. doi: 10.1111/j.1360-0443.2011.03573.x 
82. 
Bureau of Labor Statistics, U.S. Department of Labor. Food and beverage serving and related workers. 
Occupational Outlook Handbook, 2014-15 Edition. Available at: http://www.bls.gov/ooh/Food-Preparation-and-
Serving/Food-and-beverage-serving-and-related-workers.htm. Accessed August 31, 2020. 
83. 
Green LW, McAlister AL. Macro-intervention to support health behavior: some theoretical perspectives and 
practical reflections. Health Educ Q. 1984;11(3):322-339. 
84. 
Vos SC, Sutton J, Yu Y, et al. Retweeting risk communication: The role of threat and efficacy. Risk Anal. 
2018;38(12):2580-2598. doi: 10.1111/risa.13140 
85. 
Green MC, Brock TC. The role of transportation in the persuasiveness of public narratives. J Pers Soc Psychol. 
2000;79(5):701-721. 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
86. 
Green MC, Brock TC. In the mind's eye: transportation-imagery model of narrative persuasion. In: Green, MC, 
Strange, JJ, Brock, TC, eds. Narrative Impact: Social and Cognitive Foundations. Mahwah, NJ: Erlbaum; 2002:315-
341. 
87. 
Fisher W. Narration as a human communication paradigm: the case of public moral argument. Communication 
Monographs. 1984;51 
88. 
Fisher W. Human Communication as Narration. 2nd ed. Columbia, SC: University of South Carolina Press; 1989. 
89. 
Reinhart AM, Feeley TH. Comparing the persuasive effects of narrative versus statistical messages: a meta-
analytic review. 2007 NCA Annual Convention Communicating Worldviews: Faith-Intellect-Ethics; November 15-
18, 2007; Chicago, IL. 
90. 
Green MC. Narratives and cancer communication. J Commun. 2006;56(1):S163-S183. 
91. 
Petraglia J. Narrative intervention in behavior and public health. J Health Commun. 2007;12(5):493-505. 
92. 
Cohen J. Defining identification: a theoretical look at identification of audiences with media characters. Mass 
Communication and Society. 2001;4(3):245-264. 
93. 
Slater MD, Buller DB, Waters E, Archibeque M, LeBlanc M. A test of conversational and testimonial messages 
versus didactic presentations of nutrition information. J Nutr Educ Behav. 2003;35(5):255-259. FY ended before 
2008. 
94. 
Bellis MA, Hughes K, Lowey H. Healthy nightclubs and recreational substance use. From a harm minimisation to 
a healthy settings approach. Addict Behav. 2002;27(6):1025-1035. 
95. 
Eiser J, Ford N. Sexual relationships on holiday: a case of situational disinhibition. J Soc Pers Relat. 
1995;12(3):323-339. 
96. 
Bellis MA, Hughes K, Thomson R, Bennett A. Sexual behaviour of young people in international tourist resorts. 
Sex Transm Infect. 2004;80(1):43-47. 
97. 
Bellis MA, Hughes KE, Dillon P, Copeland J, Gates P. Effects of backpacking holidays in Australia on alcohol, 
tobacco and drug use of UK residents. BMC Public Health. 2007;7:1. 
98. 
Benotsch EG, Nettles CD, Wong F, et al. Sexual risk behavior in men attending Mardi Gras celebrations in New 
Orleans, Louisiana. J Community Health. 2007;32(5):343-356. 
99. 
Hughes K, Bellis MA, Calafat A, Juan M, Schnitzer S, Anderson Z. Predictors of violence in young tourists: a 
comparative study of British, German and Spanish holidaymakers. Eur J Public Health. 2008;18(6):569-574. 
100. 
Ragsdale K, Difranceisco W, Pinkerton SD. Where the boys are: sexual expectations and behaviour among young 
women on holiday. Cult Health Sex. 2006;8(2):85-98. 
101. 
Tutenges S, Hesse M. Patterns of binge drinking at an international nightlife resort. Alcohol Alcohol. 
2008;43(5):595-599. 
102. 
Pew Research Center. Social Media Fact Sheet. Available at: http://www.pewinternet.org/fact-sheet/social-
media/.  Published June 12, 2019.  Accessed August 28, 2020. 
103. 
Amir M, Sampson BP, Endly D, et al. Social networking sites: emerging and essential tools for communication in 
dermatology. JAMA Dermatol. 2014;150(1):56-60. doi: 10.1001/jamadermatol.2013.6340 
104. 
Walther JB, Pingree S, Hawkins RP, Buller DB. Attributes of interactive online health information systems. J Med 
Internet Res. 2005;7(3):e33. DOI: 10.2196/jmir.7.3.e33. FY ended before 2008. 
105. 
Walther JB, Tong ST, DeAndrea DC, Carr C, Van Der Heide B. A juxtaposition of social influences: web 2.0 and the 
interaction of mass, interpersonal, and peer sources online. In: Birchmeier, Z, Dietz-Uhler, B, Strasser, G, eds. 
Strategic Uses of Social Technology: An Interactive Perspective of Social Psychology. Cambridge, England: 
Cambridge University Press; 2011. 
106. 
Romm T. Facebook's Zuckerberg just survived10 hours of questioning by Congress. The Washington Post. April 
11, 2018. https://www.washingtonpost.com/news/the-switch/wp/2018/04/11/zuckerberg-facebook-hearing-
congress-house-testimony/?noredirect=on&utm_term=.fd98bc76ea38. Accessed August 31, 2020. 
107. 
Confessore N. Cambridge Analytica and Facebook: the scandal and the fallout so far. The New York Times. April 
4, 2018. https://www.nytimes.com/2018/04/04/us/politics/cambridge-analytica-scandal-fallout.html. Accessed 
August 31, 2020. 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
108. 
Wagner K, Molla R. Facebook lost around 2.8 million U.S. users under 25 last year. 2018 won't be much better. 
Recode. February 12, 2018. https://www.recode.net/2018/2/12/16998750/facebooks-teen-users-decline-
instagram-snap-emarketer. Accessed August 31, 2020. 
109. 
Mager RF. Preparing Instructional Objectives. Palo Alto: Fearon Publishers; 1962. 
110. 
Bloom BS, Krathwohl DR, Engelhart MD, Furst EJ, Hill WH. Taxonomy of Educational Objectives Handbook 1: 
Cognitive Domain. 2nd ed. White Plains, New York: Longman; 1966. 
111. 
Horton W. e-Learning by Design. San Francisco, CA: Pfeiffer; 2006. 
112. 
Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs NJ: Prentice 
Hall; 1986. 
113. 
Bandura A. Health promotion by social cognitive means. Health Educ Behav. 2004;31(2):143-164. 
114. 
Gliksman L, McKenzie D, Single E, Douglas R, Brunet S, Moffatt K. The role of alcohol providers in prevention: an 
evaluation of a server intervention programme. Addiction. 1993;88(9):1195-1203. 
115. 
McKnight J; U.S. Department of Transportation. National Highway Traffic Safety Administration. Development 
and field test of a responsible alcohol service program. Volume III: final results. Available at: 
https://rosap.ntl.bts.gov/view/dot/1507. 1989.  Accessed August 31, 2020. 
116. 
McKnight AJ. Factors influencing the effectiveness of server-intervention education. J Stud Alcohol Drugs. 
1991;52(5):389. 
117. 
McKnight AJ. Server intervention: accomplishments and needs. Alcohol Health Res World. 1993;17(1):76. 
118. 
Russ NW, Geller ES. Training bar personnel to prevent drunken driving: a field evaluation. Am J Public Health. 
1987;77(8):952-954. 
119. 
Saltz RF. Research needs and opportunities in server intervention programs. Health Educ Behav. 1989;16(3):429-
438. 
120. 
Seavers D. Social media statistics in the United States. Talkwalker blog. Published 2019.  Available at: 
https://www.talkwalker.com/blog/social-media-statistics-united-states.  Accessed August 31, 2020. 
121. 
Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res 
Methods Instrum Comput. 2003;35(3) 
122. 
Macefield R. How to specify the participant group size for usability studies: a practitioner's guide. J Usability 
Stud. 2009;5(1): 34-45. 
123. 
Turner CW, Lewis JR, Nielsen J. Determining usability test sample size. In: Karwowski, W, ed. International 
Encyclopedia of Ergonomics and Human Factors. Vol 2nd. Boca Raton, FL: CRC Press; 2006:3084-3088. 
124. 
Brooke J. SUS - A quick and dirty usability scale. In: Jordan, PW, Thomas, B, Weerdmeester, BA, McClelland, IL, 
eds. Usability Evaluation in Industry. London: Taylor & Francis; 1996:189-194. 
125. 
Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J 
Usability Stud. 2009;4(3):114-123. 
126. 
Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact. 
2008;24(6):574-594. 
127. 
Buller DB, Borland R, Woodall WG, et al. Randomized trials on Consider This, a tailored, internet-delivered 
smoking prevention program for adolescents. Health Educ Behav. 2008;35(2):260-281.PMC4380290 
128. 
Buller DB, Woodall WG, Zimmerman DE, et al. Randomized trial on the 5 A Day, the Rio Grande Way website, a 
web-based program to improve fruit and vegetable consumption in rural communities. J Health Commun. 
2008;13(3):230-249.PMC4376106 
129. 
Woodall W, Zimmerman D, Stapel L, et al. Systematic development of a website to reduce risky college alcohol 
consumption. Paper presented at: Society for Prevention Research Annual Meeting; May 30 - June 1, 2007; 
Washington, D.C.  
130. 
Woodall WG, Starling R, Buller D, Donohew L, Zimmerman R, Helme D. DA018575: Web-based substance abuse 
and STD/HIV prevention. Application funded by the national institute on drug abuse. 2009.  Available at: 
http://projectreporter.nih.gov/project_info_description.cfm?aid=7914379&icde=23322944&ddparam=&ddvalu
e=&ddsub=&cr=8&csb=default&cs=ASC.  Accessed September 4, 2020 
131. 
Woodall WG, Starling R, Wheeler C, Bryan A, Buller D. AI084081: Web enhanced adoption of HPV vaccine in 
minority communities. 2009.  Available at: 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
http://projectreporter.nih.gov/project_info_description.cfm?aid=7763578&icde=23322944&ddparam=&ddvalu
e=&ddsub=&cr=7&csb=default&cs=ASC.  Accessed September 4, 2020. 
132. 
Pagoto S, Baker K, Griffith J, et al. Engaging moms on teen indoor tanning through social media: protocol of a 
randomized controlled trial. JMIR Research Protocols. 2016;5:e228. doi: 10.2196/resprot.6624. PMC5147712. 
133. 
Pagoto S, Tulu B, Agu E, Waring ME, Oleski JL, Jake-Schoffman DE. Using the Habit app for weight loss problem 
solving: development and feasibility study. JMIR Mhealth Uhealth. 2018;6(6):e145. doi: 
10.2196/mhealth.9801.PMC6031896. 
134. 
Pagoto S, Tulu B, Agu E, Waring M, Oleski J, Jake-Schoffman D. The feasibility of incentivizing participation in an 
online social network weight loss program. Paper presented at: 50th Hawaii International Conference on System 
Sciences; Jan 4, 2017; Waikoloa, H.  
135. 
Pagoto SL, Waring ME, Schneider KL, et al. Twitter-delivered behavioral weight-loss interventions: a pilot series. 
JMIR Res Protoc. 2015;4(4):e123. doi: 10.2196/resprot.4864.PMC4704936 
136. 
Glaser BG, Strauss AL. The Discovery of Grounded Field Theory. Hawthorne, NY: Aldine De Gruyter; 1967. 
137. 
Krueger RA. Focus groups: A practical guide for applied research. 2 ed. Thousand Oaks, CA: Sage Publications; 
1994. 
138. 
Sauro J, Lewis JR. Quantifying the User Experience: Practical Statistics for User Research. Morgan Kaufmann; 
2012. 
139. 
Sauro J. A Practical Guide to Measuring Usability: 72 Answers to the Most Common Questions About Quantifying 
the Usability of Websites and Software. CreateSpace Independent Publishing Platform; 2010. 
140. 
Buller MK, Bettinghaus E, Fluharty L, et al. Improving health communication with photographic images that 
increase identification in three minority populations. Health Educ Res. 2019;34(2):145-158. 
141. 
Facebook IQ. Effective Frequency: Reaching Full Campaign Potential. Available at: 
https://www.facebook.com/business/news/insights/effective-frequency-reaching-full-campaign-potential.  
Published July 21 2016.  Accessed August 28, 2020. 
142. 
Buller D, Pagoto S, Bibeau J, Walkosz B, Berteletti J, Baker MK. Insights on HPV vaccination in the United States 
from mothers’ comments on Facebook posts in a randomized trial. EUROGIN 2018; December 2-5, 2018; Lisbon, 
Portugal. 
143. 
Sutton J, Gibson CB, Phillips NE, et al. A cross-hazard analysis of terse message retransmission on Twitter. Proc 
Natl Acad Sci. 2015;112(48):14793-14798. doi: 10.1073/pnas.1508916112 
144. 
Sutton J, Renshaw SL, Butts CT. COVID-19 retransmission of official communications. PLoS One. Under review 
145. 
Woodall WG, Buller DB, Saltz RF, Starling R, Stanghetta P. Uptake of web-based RBS training to prevent alcohol 
over-service. 43rd Annual Alcohol Epidemiology Symposium of the Kettil Bruun Society June 5-9, 2017; Sheffield, 
UK. 
146. 
Kolowich L. How often should you post on Facebook? [new benchmark data]. HobSpot. Vol 2019: HubSpot Web 
site; 2015.  Available at: https://blog.hubspot.com/marketing/facebook-post-frequency-
benchmarks#sm.00005x98lq12afhsyx41k4r79b6f9.  Accessed August 31, 2020. 
147. 
NOW Marketing Group. How often should you post on your social media pages? NOW Marketing Group Web 
site. Available at: https://nowmarketinggroup.com/often-post-social-media-pages/. Published 2017. Accessed 
August 31, 2020. 
148. 
9 steps to create effective Facebook groups for your business in 2018. Social Report Blog. Social Report Website; 
2018.  Available at: https://www.socialreport.com/insights/article/115005705263-9-Steps-to-Create-Effective-
Facebook-Groups-for-Your-Business-in-2018.  Accessed August 31, 2020. 
149. 
Buffer homepage. Buffer Web site. Available at: https://buffer.com/. Accessed August 31, 2020. 
150. 
Saltz R, Buller DB, Woodall WG, Buller MK, Grayson A. Refusal of pseudo-intoxicated customers by retail 
marijuana outlets in three states. 27th Annual Meeting of the Society for Prevention Research; May 29-31, 2019; 
San Francisco, CA. 
151. 
Buller DB, Woodall WG, Saltz R, Starling R. Pseudo-underage assessment of compliance with ID regulations at 
retail marijuana outlets in Colorado. J Stud Alcohol Drugs. 2016;77(6):868-872.PMC5088169 
152. 
Buller DB, Woodall WG, Saltz R, Buller MK. Compliance with ID regulations by recreational marijuana stores in 
two US states. J Stud Alcohol Drugs. 2019;80(6):679-686. PMCID in process. 

WAYTOSERVE PLUS: IN-SERVICE PROFESSIONAL DEVELOPMENT COMPONENT TO IMPROVE RESPONSIBLE ALCOHOL SERVICE 
Protocol 
202210; v.3 
 
153. 
McKnight AJ, Langston EA, Marques PR, Tippetts AS. Estimating blood alcohol level from observable signs. Accid 
Anal Prev. 1997;29(2):247-255. 
154. 
Saltz RF, Woodall WG, Wheeler DR. Does gender matter when serving obviously-intoxicated patrons? Paper 
presented at: 2001 Scientific Meeting of The Research Society on Alcoholism; June 22-28, 2001; Montreal, 
Canada.   
155. 
Woodall GW, Saltz R, Buller D, Starling R, Stanghetta P. Results of a randomized trial of web-based responsible 
beverage service training: WayToServe.org. Poster presented at: 20th Annual Meeting of the Society for 
Prevention Research; May 29 - June 1, 2012; Washington, DC.   
156. 
Buller DB, Woodall WG, Saltz R, Grayson A, Buller MK. Implementation and effectiveness of an online 
responsible vendor training for recreational marijuana stores in Colorado, Oregon, and Washington State. J 
Public Health Manag Pract. 2019;25(3):238-244. doi: 10.1097/PHH.0000000000000843.PMC6395576 
157. 
Facebook groups analytics & management tools. Grytics Web site. Available at: https://grytics.com/. Accessed 
August 31, 2020. 
158. 
Buller D, Walkosz B, Berteletti J, et al. Insights on HPV vaccination in the United States from mothers’ comments 
on Facebook posts in a randomized trial. Hum Vaccin Immunother. 2019;15(7-8):1479-1487. doi: 
10.1080/21645515.2019.1581555. PMC6746513. 
159. 
WWC standards brief for baseline equivalence. Available at: 
http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_brief_baseline_080715.pdf. Accessed September 4, 
2020. 
160. 
Raudenbusch S, Bryk AS, Cheong YF, Congdon Jr. RT. HLM 6: Hierarchical Linear and Nonlinear Modeling. 
Lincolnwood, IL: Scientific Software International, Inc.; 2004. 
161. 
Stanford University. Introduction to ROC5. Available at: http://web.stanford.edu/~yesavage/ROC.html.  
Published August 26 2008.  Accessed August 29, 2020.