Flo A. SteinFlo A. Stein, MPH
NC PIC Project Manager
Deputy Director
Division of MH/DD/SAS


Evidence-based Programs and Practices

The mission for the NC PIC is to ensure that all North Carolinians will receive excellent care that is consistent with our scientific understanding of what works whenever they come into contact with the DMHDDSAS system.

To improve the lives of clients during the current era of system transformation, North Carolina must focus on the content and quality of services and supports that are offered. Research has found that even some of the most popular and well disseminated programs are not evidence based and in fact can be counterproductive. The provision of quality services and supports involve fidelity to proven intervention models.

To facilitate guidance in determining the future evidence-based services and supports that will be provided through our public system, the Director of the Division of Mental Health, Developmental Disabilities and Substance Abuse Services has developed the North Carolina Practice Improvement Collaborative (NC PIC). The NC PIC is comprised of representatives of all three disabilities and meets thrice yearly to review and discuss current and emerging best practices for adoption and implementation across the State.

NC PIC Updates

Coalition for Evidence-Based Policy Issue Briefs

The nonprofit Coalition for Evidence-based Policy has issued two issue briefs on study designs:

Which Study Designs are Capable of Producing Valid Evidence of Effectiveness in October 2014: Randomized control trials are recommended when possible. Quasi-experimental designs are the next best choice. Additional reading materials are recommended. View the Brief

Validity of Comparison Group Designs, January 2014: This publication stated that the comparison group designs most likely to produce valid results contained the following: (1) program and comparison groups are similar in pre-program characteristics such as demographics, pre-program measures of the outcome that the program is trying to improve, and geographic location; (2) outcome data are collected in the same way for both groups; (3) motivation of program and comparison group members is similar; and (4) statistical methods are used to adjust for minor pre-program differences between the two groups. View the Brief

In addition, the website includes what they consider to be top tier programs, some of which are offered in North Carolina and other resources. A publication list can be found here.

Issue Briefs on Effective Implementation and Evaluation of Evidence-based Programs

In September 2014, the Office of the Assistant Secretary for Planning and Evaluation (ASPE), US Department of Health and Human Services published three issue briefs on elements to consider in implementing and evaluating evidence-based programs and practices.

Each issue brief is briefly outlined below:

Willing, Able –> Ready: Basics and Policy Implications of Readiness as a Key Component for Implementation of Evidence-Based Interventions:

Readiness of an organization to implement an evidence-based intervention (EBI) is considered to be essential to its successful adoption. Three components—motivation of staff, general organizational capacities, and intervention-specific capacities—are defined as R=MC2. The brief recommends that provider agencies examine their organizational readiness to determine their technical assistance needs prior to implementation and as a part of continuous program improvement.

The Importance of Contextual Fit when Implementing Evidence-Based Interventions:

Contextual fit is defined as the match between the strategies, procedures, or elements of an intervention and the values, needs, skills, and resources available in a setting. Eight elements are identified as determining the contextual fit: need, precision, an evidence base, efficiency, skills/competencies, cultural relevance, resources, and administrative and organizational support. The brief contends that contextual fit is undervalued and that technical assistance should focus on building strong contextual fit before investing in direct implementation efforts.

Using Evidence-Based Constructs to Assess the Extent of Implementation of Evidence-Based Interventions:

Five constructs are summarized in measuring implementation milestones: fidelity of implementation, competence in use, feeling and perceptions, context of the organization and community, and supporting implementation. The brief encourages organizations to evaluate the implementation process from multiple perspectives in order to identify barriers to implementation as well as their solutions. A sample monitoring plan is provided.


The issue briefs not only lay out elements to consider in implementing and evaluating EBIs, but they also outline what many federal and state funding agencies are already mandating in grant applications. Policymakers look at organizational readiness, contextual fit, and implementation and evaluation results to determine the viability and sustainability of supported interventions that applicants are proposing.


Funded by the US Substance Abuse and Mental Health Services Administration, the Service Members, Veterans, and their Families (SMVF) Technical Assistance Center has assisted members of the Governor’s Working Group on Veterans, Service Members, and their Families in developing infographics on specific topics.

Currently, two have been developed—one for Student Veterans and one for Military Families and the professionals who help them. The infographics are useful in identifying strategies and resources. An infographic on employment is currently in progress with one planned for women veterans in the future.

If you use either or both infographics, please email Dr. Fang (wei.li.fang@governorsinstitute.org) as to how you are using them. We would love to know if they are helpful and/or how they can be improved or disseminated.

Download North Carolina Support Networks for Student Veterans Infographic

student veteran infographic_Page_1

Download North Carolina Support Networks for Military Families Infographic

North Carolina Military Families infographic_Page_1










Download North Carolina Support Networks for Veterans Seeking Employment

North Carolina Employmentv Infographic_Page_1


Mark your Calendar! Next NC PIC Conference on March 10, 2015

Next NC Practice Improvement Collaborative Conference to Highlight the Behavioral Health of Military Families

On March 10, the NC Division of Mental Health, Developmental Disabilities, and Substance Abuse Services, the Governor’s Institute on Substance Abuse, and the Southeast Addiction Technology Transfer Center are sponsoring an all-day Practice Improvement Collaborative conference to highlight behavioral health best practices for military families in North Carolina.

Speakers will include: Diane Coffill, Director, Families Program, North Carolina National Guard; Steven L. Sayers, Ph.D., Assistant Professor, University of Pennsylvania and Philadelphia VA MIRECC; Gregory L. Inman Ph.D., Team Leader, Raleigh Vet Center; and Robert M. Bray, Ph.D., Senior Researcher, and Becky Lane, Ph.D., Research Psychologist, RTI International.

Ms. Coffill heads the Families Program, with staff located across the State in 13 centers. She works closely with National Guard families to ensure that they receive the services and supports that they need. Dr. Sayers offers therapeutic services to military couples and their families and has published research on integration challenges that military families face post deployment. Dr. Inman provides integrative behavioral couples therapy to couples in a Vet Center setting. Dr. Bray and Dr. Lane are collaborating with the developers of the Prosper platform used to deliver the Strengthening Families Program in twelve counties across the State. Panel members will include the speakers as well as military family members.

The conference will be held at the McKimmon Conference and Training Center at North Carolina State University in Raleigh.

Register for the Conference

To prepare for the March 10 conference, the North Carolina National Guard (NC NG), the North Carolina Department of Health and Human Services (NC DHHS), and the Governor’s Institute on Substance Abuse are collaborating to identify services and supports that are helpful to military families in the State. Respondents will also be asked to identify areas where more services and supports are needed. If you are a member of a military family, please consider providing your feedback as to existing services and support in North Carolina. The survey will be closed on Saturday, February 28. Your responses will remain confidential as results will only be reported in aggregate (as a group). Results will be summarized for the statewide conference on March 10 and will be used to plan additional services or to change current one.

Take the Survey

The Future of Crisis Response – January 20, 2015

The Future of Crisis Response in North Carolina

Follow-up to the Practice Improvement Collaborative Conference on January 20, 2015


On January 20, 2015, the NC Practice Improvement Collaborative and the Crisis Solutions Initiative sponsored a conference on crisis services, inviting four leaders in the field to present. Speakers included: David Covington, CEO and president of Recovery Innovations; Mary Smith, Executive Director, REAL Crisis Intervention, Inc.; Larry Villano, Chief Operations Officer of Terros and the Clinic Operations Director for the Choices Network of Arizona; and Becky Stoll; Vice President, Crisis and Disaster Management of Centerstone. In addition, Dr. Aldona Wos, Secretary, North Carolina Department of Health and Human Services, presented her vision for the State: to eliminate stigma; to embrace the use of technology to its fullest extent; to promote community-based services at the time and place care is needed; to provide inpatient care as close to home as possible, combined with appropriate transitional services; and to support the recovery model for all people with mental illness .


One hundred sixty-three providers, LME/MCO staff, state agency staff, consumers, and advocates attended the meeting and evaluated it highly. View a write-up of the conference from North Carolina Health News.

View Meeting Agenda


Watch the Meeting:

Part 1

Courtney Cantrell, Ph.D., Director, North Carolina Division of Mental Health, Developmental Disabilities, and Substance Abuse Services

Background and Introduction of the Day
Flo Stein, Deputy Director, Community Policy Management Section, North Carolina Division of Mental Health, Developmental Disabilities, and Substance Abuse Services

The Crisis Solutions Initiative in North Carolina
Crystal Farrow, Project Manager, Crisis Solutions Initiative, North Carolina Division of Mental Health, Developmental Disabilities, and Substance Abuse Services

Not Another Life to Lose
David Covington, CEO and President, Recovery Innovations, Phoenix, AZ

Part 2 (last 20 minutes)

The Time for Crisis is Now
David Covington, CEO and President, Recovery Innovations, Phoenix, AZ

Part 3

The North Carolina Vision for Crisis Solutions
Dr. Aldona Wos, Secretary, North Carolina Department of Health and Human Services

Building a Crisis Response Network
Larry Villano, Chief Operations Officer, Terros and Clinic Operations Director, Choices Network of Arizona

Part 4

State of the Art in Crisis Services
Becky Stoll, Vice President for Crisis and Disaster Management, Centerstone, Nashville

Panel Discussion / Q&A
David Covington, Mary Smith, Becky Stoll, and Larry Villano
Moderated by Crystal Farrow

Implementation Survey and Wrap Up
Flo Stein


Download Presentations:

Not Another Life to Lose – David Covington, LPC, MBA

The Time for Crisis is Now – David Covington, LPC, MBA

State of the Art in Crisis Services – Becky Stoll, LCSW

For addiction counselors who attended and would like continuing education credits, you are eligible to receive 6 hours of credit from the North Carolina Substance Abuse Professional Practice Board (event 15-093-S).


Suicide prevention was one of the topics addressed by our guest speakers. As a follow-up, you may be interested in an upcoming webinar series. The National Institute on Mental Health is partnering with the National Council for Behavioral Health to sponsor a six-part webinar series that begins at 2:00 pm on January 29.

This webinar series is in response to the February 2014 Prioritized Research Agenda for Suicide Prevention: An Action Plan to Save Lives developed


by the National Action Alliance for Suicide Prevention’s Research Prioritization Task Force. The plan outlines the research areas that show the most promise in helping to reduce the rates of suicide attempts and deaths in the next 5-10 years. The National Action Alliance for Suicide Prevention has also developed a toolkit, Zero Suicide Toolkit. David Covington, one of our featured conference speakers, is a co-lead on this toolkit.

In North Carolina, suicide prevention efforts culminated in the 2015 Suicide Prevention Plan, developed by the NC Suicide Prevention Planning Team, which consists of representatives from the Division of Public Health’s Injury and Violence Prevention Branch; the Division of Mental Health, Developmental Disabilities, and Substance Abuse Services (DMH/DD/SAS); and the University of North Carolina Gillings School of Global Public Health’s Department of Health Behavior. The plan, as well as an executive summary and data presentation, may be downloaded here. DMH/DD/SAS contacts for the plan are Dr. Sarah Potter (919-715-5989) and Susan Robinson (919-715-2262). A plan to reduce youth suicide may be downloaded here.

On May 12, the NC Suicide Prevention Planning Team will host the NC Suicide Prevention Summit at the NC State University McKimmon Center in Raleigh. Please save the date and look for details in the upcoming months.