Social Marketing Community of Practice #3: The Art of Evaluating Social Marketing
- Articles, Blog

Social Marketing Community of Practice #3: The Art of Evaluating Social Marketing

(Joanne Oshel)
Hello, everyone, and welcome to today’s Project AWARE
and Healthy Transitions webinar and the third in this series of the “Social Marketing
Community of Practice,” entitled “The Art of Evaluating
Social Marketing.” With all of our webinars
and office hour sessions, we do have to post the disclaimer
that the views, opinions, and content expressed
in this presentation do not necessarily reflect
the views, opinions, or policies of the Center for Mental Health Services, the Substance Abuse
and Mental Health Services Administration, or the US Department
of Health and Human Services. As you can see and most of you
are probably aware of, SAMHSA does have
the two initiatives, the Healthy Transitions
and Project AWARE initiatives. The main purposes of these
are making schools safer, increasing access
to mental health services, and promoting resilience
and equity in communities. This particular webinar is covering
both of these initiatives, so we definitely like
to welcome everybody from across the board today. (Amanda Lipp)
Welcome, everyone, again to this webinar series. I just want to give a little bit
of background to this particular series
that you all are on. So this webinar is called
“The Art of Evaluating Social Marketing,” and it’s the third of four webinars
that we are doing this year, so thanks again,
everyone, for joining. We’re really excited
about this topic and having Thomas and Brian
here who are just experts in their field
on social marketing, so are super-excited
to have them here. And I’m now just going to pass
it on to Thomas and Brian to introduce themselves
and lead us through their material. (Thomas Houston)
Woo-hoo! All right, everybody, my name
is Thomas Houston III. I am the Poverty Center Lead
at Medici Road where we really speak
to reduce generational poverty, and one of the things
that we focus on is what we’re here
to talk about today, which is evaluating programs
so that we know what works and what doesn’t work. So it’ll be
very high energy today, and I’m going to let
Brian introduce himself so you can kind of get
a little bit of a feel for what him
and Ghost do as well and then we can
jump right in. (Brian Barnes)
So, hi everyone. This is Brian Barnes,
and I am the Principal of Ghost. And I’m going to share today
some of the efforts that we’re doing in Oklahoma
with our Now Is The Time communities and share some of the results
that we are seeing. I’m looking forward
to our time together. (Thomas)
Let’s just jump right in. We’re going to have three main
overall objectives for today’s conversation. And we say conversation,
because we really want you to be able to ask any questions
that you have around evaluation. So the first objective is
to discuss why evaluating social marketing
is so important, the second one is to identify social marketing
evaluation strategies, and the third one is for us
to create talking points for you to be able to have
with your evaluation team. So what–as Amanda alluded to
a little bit earlier, we’re really into the third step
or third piece of the puzzle, if you will. We started off with talking
about the campaign: what does it mean
to actually develop this initial social marketing campaign. We moved on to have
a conversation around the target market
and making sure we’re talking to
the right people. And now we’re going to have
a pretty in-depth conversation around evaluation. And the whole reason we do
evaluation is just to be able to understand if what you’re
doing is working or not. Now, we’re actually
going to do a quick poll, so for people who are online, you can kind of answer
this question in the chat box. Where do you
consider yourselves, where do you see yourselves when it comes to evaluating
social marketing? Do you see yourselves
as a beginner, so where you’re actually only
evaluating the programming? Do you consider
yourselves intermediate, where you’re evaluating the program
but you’re also starting to take a look at what pieces
of social media work? Or do you consider yourselves
advanced or experts where you’re evaluating
everything, the programs, and overall social marketing? Let’s talk about why is
evaluation important, which is really the crux
of today’s conversation amongst the team. So I would look at it as
three overall reasons why you want to actually
prioritize evaluation. So look at the first one is,
this whole idea around we all do the work
because we’re seeking to effect some kind
of behavior change. And so oftentimes we have
conversation with our friends and they’ll say, “Do you
enjoy what you do?” And most of us will say, “Yes,” and they’ll continue to ask us
questions, but my hope is that if they ask the question,
“Is what you’re doing working?” you should be able
to answer that question. So that’s where evaluation
becomes important, because you want to know is
what you’re doing working. Also you want to at some point
increase program participation. Well, how do you know
what are the key points you want to talk about? If you’re not evaluating
your communication, it’ll be hard to know, like, key pieces
to use and sell people on, and so that’s where you want
to do an evaluation as well. And the last one is I know
all of us do this work and we would do it for free
if there was no such thing as bills, but in general, the way
things typically work is that for us to continue
to do the work, somebody has to pay for it. And it becomes very difficult to add
or secure additional funding if we can’t let people know
if this work is working. And so when you start talking about
different ways to communicate, different ways
to do social marketing, we need to be able to say
that it’s working or it’s not working, make adjustment so we
can continue to ask for more funding, and so evaluation
becomes very important for all three of these points. Now, I know we often talk–
I think it’s about 25% of you all said you actually started
to evaluate social media, and to be honest,
only up until about a year ago, the words social media
and social marketing were interchangeable. And we’re now getting
to the point where people are starting to see
the differences. Well, we recognize
there’s a difference, we should be evaluating them, and so just recognizing that if you’re
only evaluating social media it’s going to cause you
to skip or miss some of the key
program questions, and so we want to make sure we’re
not only looking at social marketing. And we actually have
a funny cartoon– well, I think it’s funny and I have a weird
sense of humor. This whole idea–
imagine you all– I know some of you
have been in a meeting at some point in time
and been– the question’s come up, “Well, what’s this big
marketing campaign idea?” And it could have been you,
not pointing any fingers but somebody said,
“We’ve got this really good idea. “We’re going digital,
we’re going to do Facebook, YouTube, we’re going to create a mobile app,
we’re going to do Pinterest.” And then somebody in the room said,
“Wow, that’s awesome. But what are we going to do
in those channels?” And I’m pretty sure somebody said,
“Well, we don’t know. We’ll figure that out later.” But you were really excited
about the channels that you were doing. And so just recognizing
that strategy is not the same thing
as a tactic. So recognizing that you
want to be in those spaces is not a strategy
to be in those spaces. And strategy becomes
very important based on this slide right here. So if anybody’s never
seen this slide before or this picture or this image, we’re really talking about
the trans-theoretical model of change that it really
becomes important as we’ve all bought
into this idea that as people make a decision to get better,
to help themselves, to change their behavior,
it’s not linear. It may go backwards,
it may go forwards, it may skip a step,
and so we all recognize if that process itself
is not linear, how we communicate to people
should not be linear either. And so we need to be able
to make adjustments, and as we make an adjustment, we need to be able to evaluate
to understand if that works so we can enhance
what works and stop doing what didn’t work so we can help
people along this process. Another chat box question:
how many of you ever had to pitch an idea, a program,
or some kind of event to leadership, and then if you have,
what were some of the questions you had to answer in order
to gain approval, okay? What can we expect? Who are you trying to reach? What are you trying
to get people to do? Expected outcomes? Number anticipated to be served? If you think about this, they’re really
asking you to do an evaluation. And so before you even get
approved to do the program, you’re being asked
to evaluate the program. And oftentimes what happens is
unless the grant makes us do evaluation, we end up talking about or thinking
about evaluation on the back end when in reality what
we want to evaluate should be part of our original
strategy as we decide in the beginning
how to move forward. I recognize that most of us
are no longer in year one, and we want to figure out
how we are supposed to be evaluating
social marketing. It’s never too late. You just change your approach
if you’re not in the beginning stages, like, just recognize if you’ve already
started to have conversations with people about evaluation, you just weren’t using
the word “evaluation.” Another question for anybody
that has kids, nieces, nephews, cousins. This whole idea or question
that kind of came up was, “Would you ever give
your child an allowance “that was promised to them
if they didn’t complete their agreed upon chores?” We don’t even have to do
the chat box for that one. Most of the answers
are going to be no. And we recognize it
should be the same way if we’re having conversation
with our funders. This whole idea which the next session
we’re going to do is around sustainability. If we want to be sustainable, we
want to have sustainable change, we have to have
proof points to ask– continuously ask for funding. And so to ensure that
we’re getting that funding, we have to have, whether it’s
proof points about who do we help, who do we serve, did we reach our
expected outcome, did they do what we were
trying to get them to do, we have to evaluate
those questions in order to ask
for additional funding. So I want us to be able to kind
of look at a couple of slides, just to really lay out
the groundwork of why the evaluation’s even
important in the first place. Now, some of this may be familiar
to a lot of you all, and typically you have
these questions, these conversations,
around the program. I want you to start thinking
about the same questions and desire to do evaluation
around your social marketing, because in theory, social marketing
shouldn’t be a standalone. It should be complementary
to the program, and so you should be thinking
about it at the same time. And so if you’re going
to evaluate the program, you should be evaluating
the social marketing as well. Let’s get into what does
the evaluation look like? When you think about
your campaign, there’s really, I would say, five key areas
you should be monitoring and if there’s any evaluators
on the phone call, I would ask that you have these conversations
with your site as well so you got to have conversations
around these five areas. The first one is demographics. So we think about this whole idea
that neighborhoods can change whether you’re
talking about gentrification, immigration, development,
or whatever, your methods
shouldn’t stay the same. You can change that. And so if you’ve changed it,
what changed? And you need to evaluate how
people responded to that change. Consumer preferences. If you think about this
whole idea that issues seen as crucial one day
may be invisible to the next. And so, I was joking– I actually remember having a pager. Now, granted, we’re talking
15, 17 years later, but things change within the year,
and so consumers are fickle. Now, around consumer products,
they’re also fickle with how they communicate with us, which dives directly
into the next piece, which is the whole idea around trends
and social conditions. So you want to make sure– ensure that you’re
not offering services or advocating behavior changes
that have changed. Now, I know a lot of this
sounds simple, but I know we get caught up
oftentimes with trying to make sure the program is working
or letting somebody else handle the social marketing,
that we forget some of the basic
day-to-day aspects about trends, social memory, et cetera, and that’s why we want to make sure
we’re calling out that you’re monitoring this
with your evaluation for the campaign. Social memory, what was
immediately important to one generation
may be incidental to the next, and so this whole idea
when you talk particularly when you talk about
minority communities, first generation,
second generation, their memory’s not the same. And so you want to make sure when
you’re evaluating your campaign you’re– particularly if you’re looking
at 16 to 24 age group, that difference from the 25 to–
actually 25 to 40 age group, and then the baby boomers
which would be, I believe, like that’s 55,
things have just changed and what they remember
has changed, so how you communicate and you
should evaluate to make sure they’re actually taking
that information in. And lastly,
communication channels. You’ll hear a little bit more about this
when you get to Brian. A lot of us were raised on TV, and at some point we’ve been
pushed over to Internet. But if you’re trying
to really communicate, you really should be using
text messaging, and so if you’re not using
text messaging, I really want you
to consider evaluating what you’re using to look at
the return on your investment. Are you getting inquiries
based off of that? And just make sure
what you’re using is actually working from a communication
channel standpoint. I want you to think back
to your campaign, and are you actually able to answer
the following questions? The first one is number
of people that you attract to your services. The second one is number
of people that you convince to adopt healthy
or socially desirable practices. The third one is the amount
of community support that you can assemble. And the last one is the amount
of influence those people you’re talking to
are able to exert through their own
advocacy efforts. And so if you’re not able to answer
these four right now, if you have an evaluation team, I would start having a conversation
around these four questions, because even though
you can change the wording, the crux or the base
of these four questions are what you should be asking
and answering about reaching
your customer bases. And then kind of taking to that, thinking about some
of the same categories, there’s your message. Does your market research–
it’ll tell you if the people are getting and paying attention
to your message. If not, you should be reframing it. Your services. Are your services effective? Are you meeting people’s needs? And I’ll let you–to read through these
but just these four, you should be able to– you should be communicating through–
to people in your target market, you should be communicating with them
using your social marketing, and then these are
the evaluation types of things you should be looking at. And if you find that you’re getting
unfavorable answers to these questions,
you need to readdress them through the program and then– and you be addressing them
not only to action but also how you communicate
with them through the social marketing. And then after–
that’s called process evaluation, which we won’t get into
for this conversation, but after you make that change,
you evaluate it again. So, again, if you have
an evaluation team, you should be having
this conversation with them about what to expect. And if you don’t, I would be thinking
through this to make it part of your strategy,
particularly around communications. The first one is how many inquiries
for information did you receive? And if you notice, only one
of these talks about social media, so it’s not just about
likes and retweets but how many people
actually engaged and asked you a question? Did you get an e-mail?
Did you get a phone call? If you had an event,
did somebody stop you? Did you gather that information?
How are you collecting it? The second question is
how many new followers did you gain on social media? It’s great if you get
500 likes a day, but if those 500 likes
come from the same 10 people, you’re not really improving,
so you need to be able to measure how many new people
are hearing your message. The third one is
everything you do should be sending
someone somewhere, and if you have a website, they should be sending them
to your website. So you want to be able to evaluate how many times did somebody
visit that website, and then on top of that,
not only did they show up, on what pages did they visit,
but how long did they stay? If you’re evaluating that,
you can now figure out what types of things
that they want to hear about, what they want to see about,
and you can enhance that. If you know that they went
to the landing page, stayed for ten seconds,
and didn’t visit anything else, maybe it’s time to change
the landing page. If they went
to your events page but nobody clicked
for more information, maybe your event page
isn’t robust enough. So that’s the kind of things
you want to be– that’s why you want
to be able to evaluate that. And then lastly, how many people
are coming to your events, and more importantly,
how did they hear about it? If everything is word of mouth
but you’re spending $50,000 on radio, your radio is not working. Or if you’re spending
no money on radio but everybody
just happens to walk by and word of mouth’s not working, maybe it’s time to look for some
other ways to communicate. So that’s just really four types– the basic four outputs you
should be looking to evaluate. Lastly, I want to talk about how do I want you to actually
approach evaluation. And so whether you’re doing
with your evaluation team, or you’re trying to figure
out how to do it on your own in-house, I want to give you
a couple of tips to, at least as a starting point, or even if you’re in that intermediate
way to move forward. So the first one is surveys
or intercepts, and we can probably get
you guys some examples if you don’t have– if you’re not sure
what an intercept is. The next one is consumer panels. Most of us are actively talking
to a regular group of stakeholders but instead of having, like,
a one-time focus group, which is what most of us do,
we talk to them or we have access to them
on a very consistent basis. Let’s go ahead
and create a group which will be called
a consumer panel and then get regular feedback
sessions from them instead of doing it one time. That way you can start measuring
over time the type of feedback that you’re getting
and you can be more consistent in how you execute
based off of that feedback. Can we go to the next slide? I also want you to look
at how you’re monitoring your overall efficiency. And so I want you
to focus on four areas. The first one is your area
of communication. And this is really idea around
if you’re not seeing growth in inquiries, there’s probably–
start having conversations that you might be
in the wrong place. So if you’re spending all
of your time on social media but you’re not getting any inquiries,
then maybe social media– or specifically you’re in the wrong
social media channel. It might be time to change. Or if you’re doing everything
word of mouth but you’re not getting people
continue to ask for services or make inquiries about services, it might be time to move
beyond word of mouth. Second, I want you to be evaluating
the efficiency of your target population. What if you have
the right message but you’re talking
to the wrong person? So, oftentimes we talk about
going directly to the consumer. Well, maybe the message
is not getting through, so we need to be talking to influencers
of those consumers. Or it could be vice-versa. So you want to make sure
you’re measuring that. The third one is really tied
to the number one, which is whole idea around
communication channel. So it’s not just measuring
the channel of how you’re talking to people. And then lastly, the idea
around competition. I want to ensure that you’re
analyzing your feedback to make–
to see if we– that you assume
the correct information, so not only just getting the information,
but analyzing it as well. So, we go to the next one
which is I’ll– be time for us actually
to introduce Ghost. I’ve given–had a opportunity
to kind of give you kind of these three key steps
of why is it even important and then being able to move into
identifying the strategies and giving you some talking
points to have with your team. Now I’m going to give you– Ghost will give you
a opportunity to talk– walk you through a case study
of different action and what this looked
like in real life. (Brian)
Hi, everyone. So this is Brian Barnes,
and that’s exactly right. Everything that Thomas
has been talking about, we talk about in our Now Is The Time
community in Oklahoma. And so this is an opportunity where we want to share
what we’re doing, and we’re hopeful beyond
this webinar to reach out to other Now Is The Time communities
and talk about best practices. Let’s talk about what’s working
and what’s not working. So we have just finished
year three of our engagement. We’re just at the beginning
of year four. And I want to show you
what we’ve done to date, and I’ll go through
it really quickly, and then I want to share the results
that we’ve been tracking. Now, I want to preface. We’re tracking and what
I’m presenting are results specifically associated
with our social marketing, so, guys, there are surveys
about the experience and we have consumer panels in the form of service provider
monthly meetings that are outside
of the evaluation components that I’m going
to talk about today, but they do exist
within our community. Again, this is just a closer look
at the practices and findings that we’ve implemented
in Oklahoma. And our background? This may be similar
to your community, it may not be, but in Oklahoma, suicide is the second leading
cause of death for young adults age 16 to 25. Homicide is the first. And so the goals,
before we even started year one, the goals were let’s help
young adults recognize these issues. Let’s connect those that have
a need with services and let’s also work
to decrease the stigma of shame associated with these issues
in our community. And so we started
with discussion. We started with research. And there were
some great materials that were already provided,
’cause I don’t know about you all, but we didn’t have, like,
a huge research budget. We had to work with what we had. And there were some great
resources made available to us through work and reports
and white papers that had been published
by SAMHSA, by NAMI, and then we also looked
at some of our resources in our world of advertising
and marketing and we looked at Nielsen
and Common Source Media to get an idea of how our audiences
really consume media. So we did do some
qualitative research through an online survey
and we surveyed providers and young people
from our three ONIT counties and we combined
the results of the survey, these takeaways,
with our takeaways from existing research materials
and all of that was put together
into a discovery document that we shared and we discussed,
and we used that discussion to create our social marketing plan
for years one and two. So we reviewed
existing research, we conducted an online survey
with providers and young people, and then we got together
and we talked about it. We discussed the findings
and takeaways, and that’s what we used
to create our marketing plan. And during that discovery– and I know this might be
a little bit small– we looked at both Nielsen
and Common Source Media. Nielsen said–
and both had– both have surveys. Nielsen had 13 to 24,
Common Source had 13 to 17. It wasn’t our ideal age range,
but it was really, really close, guys. And Nielsen said texting is
the most preferred method for young adults
age 13 and 24. Common Source broke it down even more
for 13 to 17 year olds and they said 68% of 13
to 17 year olds preferred text, 51% say social’s up there with their fave,
30% email, 19% talk to each other
with instant message, and 11% through online gaming, and it went down
and down and down. Twitter was very low
on the list. But for texting,
13 to 17 year olds, an average of 3417 messages
per month. The next age range, 18 to 24, 1914 messages texted per month. Text, text, text. All right, the year one, it was about establishing
that foundation and deciding that we
wanted to tell a story and how to tell that story
to start a conversation. And so we worked from
that discussion to fit together a concept
and visual identity. We talked about a plan so that
we had a digital presence, how we were going to talk, what our tone was going to be
on our social media channels, and our website where we could– where we could house all of
our resources and information for our audience to find. But we also had
service providers that were very plugged in
with the community and where they interacted
with young adults in transition, our target audience,
and there was an opportunity for a physical presence. And I just mean posters
and printed collateral, your usual suspects,
where they already interacted and had had a presence physically
in our counties. And then also media. We wanted
to create some tools to help our local
media affiliates. We wanted to make it easy
for them to help spread the word
about our initiative. And so our concept,
visual identity, it was a logo mark,
graphic standards, and our campaign messaging. We changed Oklahoma
Now Is The Time. We changed our acronym
to be represented as ONIT, Oklahoma Now Is The Time. And that helped to solve
an issue where we had people pronouncing
the same initiative two different ways and that was one of the things
that we had to address that we discovered
doing that research. Let’s get everyone talking
about the same initiative and pronouncing it the same way. Colors and typography. We chose them with intention,
and we wanted to use those consistently to help build equity
in the initiative. And we found
that anonymity was something that was important
to our audience as they were deciding whether
or not this affected them. And so we said,
“All right, well, “whenever we use photography,
let’s only use photography “whenever we have it with– “associated with a story where
there’s a positive outcome. “If we’re just talking about
the statistics or how to identify, “let’s maybe not have “that image of someone’s actual peer. Let’s maybe think about
an illustrative way to do that.” And we also created a slogan
and hashtag, “I wish you knew,” that we promoted in materials. So that’s what we said is
in our area the best way to find out
if a young adult is experiencing those issues is
to simply ask them. And so we wanted our materials
to help prompt that question and to get that response. So I want to show ONIT,
we have our mark in the left-hand corner, our
color palate is the next column where we have licorice,
white sugar, grape soda, limoncello, limeade,
and berry bubblegum. Told you, we’re not–
we decided that we were not going to go black and white. We were not going to go
dark heavy colors. We were not going
to present information in a sad, depressing way
that perpetuated stigma. And then we had typefaces
that were going to be used, again, all used consistently
in our different channels so that they all reinforce
and help to build equity. “I wish you knew”
to prompt the question. Our contact info
and how to direct them– and how to direct them
where we wanted them to go. And then a series
of illustrations that I’ll show how we used
in our materials where we do use those
with statistics instead of an actual photo
of someone that’s experiencing maybe some of the negatives. And in our digital presence, we wanted a site
that engaged young adults and social content
that pushed the initiative with consistent reach
and frequency. Reach and frequency is something
that I talk about all the time and over and over
and over again. So we also designed
everything digital with mobile devices in mind. It’s where the majority of our traffic
was going to come from. We estimated 60%. It was close.
It wasn’t quite 60%, but close. We also knew that we wanted
to capture stories and include
a self-evaluation quiz so that if we got someone
to our site it wasn’t just bombarding them
with information. We wanted to give them
something useful to do. And so we had an approved quiz
where they could answer some simple questions
and they were presented with a response that said, “You maybe could benefit
from talking with someone,” or “This looks fantastic,
and you seem wonderful.” We also included an interactive map
with all of our providers so that we could help our viewers
connect with services. And we developed our social media
content two months at a time. We did not have someone
that was designated that could just stay day in
and day out posting. We wrote our content
two months at a time and we scheduled it in advance
so that we could set it and let it go. And our posts were really viewed as a replacement
for traditional ads. We didn’t have media dollars. We have social content. So we were and are
most interested in measuring those social media channels
for our reach and frequency. So our digital presence. I’ll share with you guys
in the chat. If you want to go
check it out yourself, our website is I’m putting that in
the chat window right now. It’s–our website was designed
so it’s responsive. It looks great
on any size screen and it includes
that yellow bar top, which if you’ll click it,
it takes you to an interactive map of providers across our state. Not just our three counties,
across the entire state. We have a series of video stories
of young adults and service providers
to help show young adults who may be experiencing
mental health or substance abuse issues, what it would be like
to reach out. What that other–what that end result
might look like for them. We wanted to make it
as approachable and non-judgmental
as absolutely possible. You’ll also see if you
go on that site, you’ll see
the self-evaluation quiz, and I would suggest
that you go ahead and take it for yourself
just to see how it works. And again, our scheduled–
our social posts scheduled in advance. Physical presence. We have printed collateral,
those posters and brochures, and we also created– we also created a toolkit
for media outlets that included a press release
and visual assets. We wanted to make it
as easy as possible for a media affiliate– if we could get some interest,
we wanted to make it as easy as possible for them
to run with our stories. But a series of posters
for sure and press release. And our poster series
broke down the funds to look for, the categories of emotional,
psychological, behavioral, physical, and psychosis. And in a follow-up, we’ve offered
to share the materials that we’ve done in Oklahoma, ’cause we want you
to see what we’ve done, and we want to see how it
compares to what you’ve done and figure out what works
and what doesn’t work. But we did one series
that suggested what could be a pretty common experience
or the way that a young adult might experience
one of these issues to ask them to think about it. And then we paired that
with a statistic, again, from great research that existed
for Oklahoma specifically. We were able to tie a stat
to each of these areas to help communicate how
prevalent this issue is. What you don’t see: you don’t see black and white
photos of kids in tears or in corners
or depressed and sad. We want to present
information straightforward without sugarcoating it
and also without judgment. And so that’s why you see
those illustrated elements, you see our colors. You do not see the hallmarks
of this is a bad issue, and we want you
to feel bad about it. In addition, we had a brochure
that were distributed along with areas of the location
that included the funds to look for. All of this information
is also on our site and we wanted
to drive them to– we’ve got a phone number,
we’ve got our website, and we’ve got links to our
social media channels as well. So it’s just framing this issue
with our young adults in mind. Again, we want to make it
easy for media outlets to help spread the word,
so we wrote press releases, created them as templates
for our service providers, and provided visual assets
so they could pass them along to their local media affiliates. And so this is just an example
of the press release itself and some of the visual assets
that we included. That was year one,
the foundation. In year two we wanted to produce
a series of web videos where this time we got
to bring in our actual stories. And I’d asked for stories of success
in each of the counties. And so we have
a total of three stories for each of our three counties for a series of nine videos,
guys, that are really great. They’re young adults who are
so brave and so honest about sharing their story
and their experience and how they got
on the other side, and our service providers
were wonderful and warm and talked about their experience,
so it just– it was just an incredible,
incredible experience. So we captured and created
a series of videos. We also successfully integrated a texting platform
into our campaign. I don’t know about
each of your communities, but here we have some
serious compliance issues that really limit
what we can and can’t do. One of the big hurdles,
one of the big barriers to our texting platform here
were we could never capture an individual’s phone number. In every texting platform
that’s out there, that’s exactly what
it’s designed to do. So we had to find a provider that would scramble
the numbers for us so that we never had
an individual’s actual cell phone number
so we couldn’t identify them. Then we wanted
to continue documenting and sharing content
on our channel. So that was year two. In terms of the texting platform,
I would like to ask– sorry, in our ONIT videos, all of these are on We have a series of dying. Please, oh please,
check them out. And guys, you’re not going
to hurt my feelings if you want to stop me,
if you want to add something or ask a question,
just go right ahead. Our texting platform, I would like to ask each of you
that’s in attendance if you happen to have
your smart device with you, text “I wish” to 55155, and you’ll see exactly
how we programmed it. We programmed
an autoresponder to reassure individuals
that help is available, and it directs them directly
to our website and also includes
phone numbers to call for help. We were not allowed– ideally, what we wanted to do
with the texting platform is to have a service provider on
the other end that could engage right when one of our
target audience reached out. Currently, we are
not able to do that. So the autoresponder
was our compromise. That’s what we had to do,
but I want to stress, and I’ll share some
of the results of this. I want to stress to be
effective marketers, we must be able to meet our
audience where they want us to. And then year three, so the year
that we just concluded or the year that we’ve
just wrapped up. We want to encourage
broadening the conversation by asking people
in our communities to share their truth,
their “I wish you knew…” And to do that we created
a series of community chalkboards and we went to our counties and we talked to our
service providers and we did installation videos on how to install these
in different locations, but we said,
“Let’s just ask people to, “in an anonymous way,
share their story, their experience, with mental health
or substance abuse.” And we cautioned,
we told the service providers, “Guys, expletives happen. “It’s a chalkboard,
so you’re going to have “to have someone
that’s able to monitor this “and to wipe away if
we do get an expletive “or we get something
that’s inappropriate, we have to be able to erase it.” So when the chalkboard fills up,
we want our service providers to photograph it
and document it. That is gold for our
social media content. And once it’s filled up,
we want them to erase it and start over again
so that we can get as many of these anonymous
confessions or stories as possible. To that end, we worked
with our service providers and created the chalkboard kit,
and we actually created and packaged kits that we sent
to our service providers along with these instructions on how
to make their chalkboard wall. And–so that’s the year three. We finally got our
community chalkboard kit. They’re starting to go up
and we’re starting to capture them
and they are amazing. They’re everything
that we hoped for, and I cannot wait to capture
and share what we get. But as we look at measuring
results, digital tactics, and especially social media, what’s cool is they’re
developed with metrics built right in so that we
can track engagement. On our website, we get to
include Google analytics which is free and allows us
to create real-time reports. Our social media dashboards,
they help us track and reach and identify trends. Our texting metrics, they are
currently limited by HIPAA but we are–
because of that we’re only able to track
number of texts only. But we can still
track something. And of course, in Oklahoma the metric
we’re ultimately hopeful for, like, and the one that
I think truly, truly counts at the end of this is looking for a reduction in the number
of suicides in our ONIT counties. All right, but next when
we report year end, we’ve got Google analytics,
we’ve got our social engagement, and we’ve got our
number of texts. So, Google analytics. Guys, again, free. If you have a website,
you can add Google analytics into your existing site. We said 60% were going to
come in off mobile devices. Well, we were a little bit off. It was 45%, but still almost
half coming on mobile devices, 55% coming in on desktop. Our new visitor traffic, 80% of visitors
to our site are new visitors. We have 23.44%
that are returning. So this is–
we had a soft launch in April, an official launch
in June of 2016. So June of 2016
through May 31 of 2017, 2713 sessions, 5357 page views,
2 pages per session. On average, our viewers stayed
on our site for 2 minutes, 2076 unique users,
and we have a 63% bounce rate, but I don’t want to count
the bounce rate as a negative. Bounce rate just refers to someone
that goes to your site and they just stay on one page
and then leave. Well, if they go
to our home site and they find the provider
that’s nearest them or they watch a video or they
take the self-evaluation quiz and that’s all they needed
and then they bounced, I still don’t count that
as a negative. But just to let you know,
63%, they came to, whether it was the home page
or any other page, they looked at one page
and then left. Social engagement. So, guys, I do have
some numbers to show. We don’t have the final numbers
for this fiscal year, but I wanted to share on average we have 725 that we reach
on Facebook per post, 653 per average–
average reach per tweet. From April through December– or, I’m sorry, from June of 2016 to– through December 31 of 2016, we reached 19,043 people with our Facebook posts,
9331 on our Twitter feed. Our highest engagement
we saw this year, and it’s whenever we posted
the first photographs of our community chalkboard
that was filled out, 3187 from that post
and that was not promoted. This year we will promote, and I expect those numbers
to grow astronomically. And then texting, guys,
I fought really hard for this. All of the research
shows texting is where this audience
wants to be engaged. So we had a total–
total texts in, total people that use this,
1172, but our monthly activity
you can see it spiked right after we launched
and then it cratered. It flatlined. Well, I don’t think that
the research is wrong, I do think that this audience
wants to be engaged on text, but I think what happened here–
and this is my opinion. I don’t have evidence
to back it up. But I believe that
the experience that they wanted was two-way texting,
not an autoresponder. That’s why. And so I’m sad to say
that one of the components that we fought the hardest for, the way that we’re able
to currently use it did not work, and so the texting platform
will not be a part of year four moving forward. But I want to stress again, if we
want to be effective marketers, we’ve got to figure
this part out. We’ve got to figure out how
to reach our audience where they want to be engaged. And guys, in a nutshell, that’s
what we’ve done in Oklahoma where we have
an incredible project director, we’ve got an incredible
youth specialist, and we’ve got amazing
service providers. (Thomas)
Actually, I do have one question, Brian. I love what you presented, and one of the things
that I would love if you could maybe expand on is
the whole idea around text messaging, ’cause it got me to thinking
and I wonder if you could maybe, like, talk to or share
what you saw maybe as the best piece about
using the text messaging. (Brian)
It was a hard-fought tactic to get it included
and whenever we launched and we saw those
first couple of months where the numbers spiked, I was already patting myself
on the back and congratulating all of us
that we were so brilliant. And then to see it plummet
was just sickening. We had to own it. We had to look at those numbers. So, like I say, I know
that the research is good, and I saw where
there was engagement and really organically it was
growing, growing, growing, and then all of a sudden it’s just– you have to face whatever’s–
numbers don’t lie. Numbers don’t have emotion. And whenever we saw
those numbers go down, we had to report it,
and we had to talk about it. But I just have to believe
that we had with that early engagement,
that that was viable. That was the way to go, but not having that
two-way conversation, that two-way component, the fact that it was
an autoresponder was very quickly viewed and dismissed as disingenuous or, at the very least,
not what I wanted. Or else we would have seen
those numbers continue to grow. (Thomas)
The team on the phone– a question based off of that. So let’s say you all
had hired Ghost, all of the sites
had hired Ghost, and everyone had done
text messaging and you got the same answer. As I said, we’ve each of us done
the evaluation work and we all gave
the same exact answer which was saw great spike
and then you saw it flatline because potentially there was
nobody on the end doing two-way messaging. What are some of the kinds
of questions you guys asked based off of that evaluation
to maybe change how you move forward
text messaging? What are the kinds of questions
you would ask in order to help you
change your process of using text messaging
moving forward? (Brian)
And this is self-serving but I want to ask,
I know that, so not in Oklahoma that we know that
there are other initiatives, they are able to have
this two-way engagement, and I want to know how. Why either compliance
or restrictions are different for them
than they are for us. (Thomas)
And I may have the answer for that. I’m going to tie it in to– it looks like Julie Murray
had a comment about how they’re using
text messaging. So Julie was
basically saying that– if I’m reading this correct, Julie, it looks like you were
using two-way communication but your clientele’s
about 50 people so it’s not too overwhelming. But on top of that, you’re using
it as a form of social marketing and not necessarily as a way for
polling or sole way of communications, i.e., you’re using it to inform
people about events, et cetera. And so one of the things
that I would say is the next step if you will is, you honestly have to look at
your budgetary constraint. Can you hire a person
to be there live as your two-way communicator
or can you create or add– I hate to say this– add responsibilities
to two or more people at– that currently work with you
and that’s their role? If the answer is no, then do you remember
one of the things that we talked about is when you’re evaluating, you’re
evaluating things like drop-off. You’re evaluating new
or increased participants and so if you can’t do two-way, one of the things that you
should be looking to adjust is content information
that you’re sending. So the way Julie is
using it as event, I would recommend that if
you guys haven’t tried it, typically when you send
a text message, you can also add
a link at the bottom. You can also do
A/B testing so maybe one group
gets one message, another group another message, and so increase the content
that you’re sending, recognize that you
can’t get two-way back but now you’re giving them
more information to read and you’re always linking
them back to something, whether it’s
your Instagram page, your website,
or whatever. And then you can also test
which communication they’re responding to. Is it they’re responding
to an event more? Are they responding
to a new hint or how to handle some type
of a mental health concern? And then you can kind of use
that consistent communication as a quote-unquote form
of “two-way communication.” I’m going to read this. She was thinking that Oklahoma
was reluctant to use texting due to HIPAA, and her question is
she’s wondering if other states had to implement any kind
of warm or hot response. If so, how’d you do that
with HIPAA limitations? And then I’ll throw in
two more pieces, since we’re just on a text
messaging conversation. And I don’t want anyone
to feel like that we have to talk about text messaging but since we’re on it,
I have two more pieces. The law changed, I want to say
about 10 years ago or so, and in theory, text messaging
is a form where you really shouldn’t be reaching out with
quote-unquote “promotional information.” And the information that we’re
giving out is promotional information, and so we want to make sure
it’s not unsolicited. So if you’re thinking
about your overall plan, your overall
social marketing plan, a component of that
should be collecting emails and phone numbers, and if you’re collecting,
it is no longer unsolicited. And so you have
the opportunity to– when you’re using
text messaging, you’re within the confines
of the law. The other thing that I would say
is when you’re talking about how to communicate,
HIPAA, as long as you’re, particularly using data,
high-level data, not low-level
individual-level data, it allows you to kind of
get around the– this is, I think, when you
start combining your knowledge of program with your knowledge
in social marketing. The same rules apply
with social marketing that you would use for sharing data
with the program. And so as long as you
follow the same rules, you’ll be fine with
your social marketing. Looks like there was
probably a question around frequency of evaluation. And I will probably look at it
from a standpoint of where you are
in your process. If you’re in your fourth year,
you’ll probably just want to do an outcome evaluation to be able
to say these are our results. If you’re in year two or three,
and you know you want to potentially make the change
for the following year or if you’re doing–if part
of your social marketing is very interactive, and you have the ability
to make a change at the six-month timeframe, then you would do
a process evaluation which would allow you to basically
change how you’re communicating, and then that’s something you could
do at the six-month mark to make sure the last
six months are different. Or you can do it
at the end of the year, so your next year
is different. That kind of really depends on
where you are in your process. Will you only be
looking at outcomes, or would you be looking
at process as well? Any more questions
from the team? (Amanda)
If you can share any experience on once you’ve put together
and conglomerated your data so you have outcomes
and some form of evaluation about what you do next and how you start to evaluate how– who to show that evaluation to, the communities, the members,
to funders. Can you talk about
that a little bit? (Thomas)
Sure. Want to go first, Brian?
Then I can tag on? (Brian)
Oh yeah. I can speak to it
on our side. On our side, like I say,
we meet every month. We have service
monthly provider meetings where we get to share
kind of what’s going on and we get to share successes
and challenges. But at the end of each year,
we do wrap-up and we do reporting before we
plan the following year’s scope and so at the beginning is
whatever we really talk about we can pivot and add
any combo we want but that’s what we really plan
and talk about how and where we’re going
to make changes if we need to. Formal reporting
is done annually. (Thomas)
Outside– in terms who needs to see it. And so it really depends on,
basically everybody needs to see it but for different reasons. And so I would make–you could
almost really make a one-pager which would highlight charts
of improvements, decreases, basically all the key
information that people want to see. And that really should be based
off of that first conversation you had 12 months ago,
six months ago, which said, “What are we trying to achieve?” Which is who we trying to reach,
how we get them to change, are we trying to get them
to come to events or whatever? So whatever questions
you create in the beginning, those are the result that you
want to be able to show. If the answer is zero,
then that’s– I guarantee somebody’s going
to ask a question about that but what you don’t– you essentially want to be able
to answer the questions that as a team
you guys assigned at the very beginning
of the process. And so now if those questions
are around results, that you want to ask for funding,
then funders need to see that. If you saw some results that
clearly say, you know what? We could make
a bigger improvement but the policy needs to change. Well, then, probably somebody
in city council mayor’s office, state’s office,
needs to see that. If it’s something
that shows improvement and you want to get
the community more involved, well, most of you
probably have like a Now Is The Time community team
where you’re bringing other providers or
community members in on a monthly or quarterly basis
to have conversations. That’s where you
would show them that data. So everybody sees it,
it just looks different. But at a minimum,
you should be answering and showcasing the answers
to those original questions you created and then formatting
the chart and verbiage depending on the audience. But everybody needs to see it. (Amanda)
That’s very helpful information, Thomas, especially as grantees
think about current stakeholders and how to really show
the case for what’s working or what’s not working and how
to have those conversations. Do folk have any feedback
or comments or experiences with that next step? (Thomas)
I would want to say in terms of the next step
and how you show it, I would do one of two ways. The first one is get
your evaluation team to do the long version report. You’re going to need it,
SAMHSA– somebody’s going to need it,
so go ahead and do it. On the flip side,
everybody’s busy. You’re probably
going to get 15– no matter how long
the meetings are primed for, this one included, you’re
probably only going to get 15 minutes of focused attention and so you want to be able
to use those infographics, those graphs, those quick
two- or three-sentence summaries to showcase what the images
are trying to say and then create a one,
maximum two pages to highlight– I mean, to showcase
the highlight of that longer report. And so that one or two-pager
is what you can– and when I say two-pager, I really mean one page
front and back. And so you can hand that out,
email that to anybody, or if you’re in a meeting, have a conversation
and then somebody says, “Hey, I want more,”
you have the longer report but you’re going to be– you’re going to have
to do long anyway, so let somebody do it, but what you really going to spend
your time developing is that short one-pager,
or two-page front and back, that you can basically have
on you at all times to be able to have conversations
with people about the program. The other thing I would say
about that is it’s okay to fail. Like, people always say
there’s no– all press is good press so bad
press is really good press too. Well, bad numbers are good also. If you originally set out
to see or to show that you are going to be able
to make X improvements, and after a one-year evaluation
you didn’t meet that and you aren’t even close,
well, that’s important. It’s more important
to say you didn’t meet it so you can change so you’re
not spending five years doing the same thing
and not helping anybody and people will appreciate
the fact that you said, “We thought we were
going to do X. “We ended up
30 percentage points lower, but now our recommendation
is to change and go this way.” So, numbers are good. Numbers work whether
they’re good or bad, you just have to show them. (Brian)
I just want to second what Thomas just said. Like I say, the–it wasn’t a fun
day to report the numbers where we showed the spike and then the dramatic decline
in texting. And what would have been much
worse is to have sat on that and then at the end
of a two- or three-year period to have shown hadn’t had– not addressed it or made
a change or somehow correct. So, like I say, that’s true. You experiment, you try. Some things work,
some things don’t. (Thomas)
Because remember, the social marketing is supposed
to augment the program. And so through
your social marketing, you can show what
data didn’t work, you can now have conversations
around changing the program, and that might be you need
more funding for an FTE. You might need to change
Wednesdays to Fridays. But if you’re not having
that conversation, honestly, it starts with
the people you’re talking to and the best way
to reach them is through your
social marketing campaign. You have to do the work and then
you have to evaluate the work so you can turn around, and I say work,
really to collect the social marketing so you can change the program. (Amanda)
Even if you have bad numbers, but have a recommendation attached
to those quote-unquote “bad numbers,” how that really can show
the community that A: you’re listening, but B: we have a plan
from these bad numbers how we can improve them
or change the course of action so we can reach
folks in different ways so that trial and error, whether
it goes bad or good, is always– always has a learning lesson
attached to it and a call to action from that. And I’m wondering
if folks on the line or who are engaging through chat have any resources
that they want to share or that they want to showcase that you or one of your
partner agencies has that you feel
is a good example. Or maybe not a good example,
but you want to share it. This can be a time
for some peer exchange and peer learning
amongst each other. All right, Julie shared that you
use email, text, snail mail, I love it, and Snapchat. (Thomas)
And I want to throw in one piece of information
for Julie, ’cause she’s doing email, texting,
Snapchat, and snail mail, and they’re doing
something in Maryland, that I would encourage all of you to do
is collect the information. You can’t evaluate it if you’re
not reaching them. You won’t reach them
if you’re not collecting how to reach them. So every time
you have a meeting, every time you’re
in the community, a friend of a friend, like, you should be collecting
their user name. You should be collecting
their cell phone number, even if it’s going to change
once every three months still collect it anyway, because you can’t reach them
through these different channels if you don’t have a way
to reach them. And then also, we talked
a little bit about texting. So that’d be one group
gets one thing, another group gets another, you can test who clicks the most
based on the messaging. If you don’t have
their number or their email, or their user name,
you can’t test. So the one key piece about
Julie and Maryland’s success is they’re collecting that
channel to reach them, the number, the email,
or whatever. So you can communicate,
so you can evaluate. (Brian)
I’m going to have to trademark that, CCE:
Collect, Communicate, Evaluate. (Amanda)
I wonder, too, Thomas, you’ve got me thinking,
and I wonder if you could get something to download a PDF
or to enter an art contest, an online art contest. You often first have to enter
your phone number or your email, so I wonder how folks are
thinking outside the box as far as that initial
engagement, like, why should I as a youth/young
adult give you my email? Why should I give
you my cell phone? It’s all, like, potentially very
personal contact information, so what’s the win-win situation? What are they going
to get in return for giving their
contact information? (Thomas)
One of the things that we tell everybody is,
you know, everybody’s used to hearing,
you know, “Nothing in life is free.” So we’ll give you
whatever you want, but you have to give us
something in return, so you don’t get
a promotional item, you don’t get
this downloaded PDF, you don’t get this free song
unless we get something, and the something we
typically want is an email, a good way to contact you,
or for you to post something on your social media site. And then that–there’s
that whole ability for you to give them a communication, but also still being able to collect
something from them as well. (Brian)
It’s different than our Now Is The Time initiative, but for many of our other
clients in different sectors, that’s the goal
of a lot of our tactics. It’s simply to capture
that information. You guys, solid email addresses
that are vetted are gold. (Thomas)
And particularly with your age group, I literally have the conversation
often with transition-aged youth and they’ll say,
“Well, how is it that Apple “can do such-and-such
or how is it they can, like, upgrade and download
without me asking?” And I was, like,
“I guarantee you “when you first bought
that phone, “you didn’t read
the three pages of text, you just clicked, ‘I accept.'” Well, that age group’s so quick
to get to what they want, they’ll click accept, and it’ll
take them a while to opt out, but you’ll get the first accept. Being able to collect
the information from them is pretty easy to get it now. Then it’s up to you
to give them content so they don’t opt out later. But that first “Gimme your email,”
or whatever, just to get whatever
the download is, you can get that, you just have
to create the system. And then guess what? You can evaluate that. Brian, if you have time,
can you talk a little bit more about that process-wise? I know oftentimes our
Now Is The Time team members are crunched when it comes
to dollars for social marketing. Can you talk about process,
not from a money standpoint, but from a resource
and time standpoint? How difficult,
if it’s difficult at all, to set up those systems
to be able to collect data from our youth? (Brian)
That’s a layered question. Depending on what
your end goal is, different ways to support
whatever your end goal is. Difficult to fit it
in the context for Now Is The Time
specifically, just because I’m so wired
or conditioned now to not collect
that individual data. And so that’s why, for us, it’s
just that frequency and reach. Those are some
of our key metrics, because I can’t capture
their individual email address. I can’t capture their
individual phone number. But with other clients,
for instance, our county health department,
we’ve developed an app that we’re getting ready
to launch whose sole purpose is to help
build their database where, just like you say,
the app is free, but you’re going to opt in
as soon as you download it. (Amanda)
I wonder, Brian, Thomas, if you compartmentalize these
different audiences and emails, like, for example, you have
your email listserve and then you have your Snapchat
user names and audience and you have your
Facebook users and audiences. And do you find that there’s
a lot of crossover with people across the mediums, or do you really market
and promote resources, services, what have you, to each
of these individual platforms and the audience
relative to that platform? (Brian)
If you don’t mind, I want to take that one
for Oklahoma. There is a lot of crossover. In fact, the one social media
channel where I would say is the exception is Twitter. Twitter in our research shows
that that’s not a viable channel for our target audience,
our young adult, but for their family
or for their caregiver and their service provider, it is. So, like I say,
so that’s one outlier, but other than that there’s
quite a bit of crossover. And so we do have messages
that are tailored for our young adult,
messages that are tailored for their primary
caregiver or parent, and messages that are more
community or broader based. But, really, it’s those
three audiences in that order and a lot of overlap
in all channel. (Amanda)
I wonder, too, just to layer on top of that,
maybe to differentiate between, let’s say, on Facebook
you have maybe two audiences. You have the youth
and young adults and you also have the families. Are you then
sending out two different– but the same message
but in different lingo, if you will, to the parents or caregivers
and then also to the youth? (Brian)
The answer to the first part is they’re all for consumption
by all audiences on our channels, so the way that we differentiate it is
if we’re sticking to our young adult, it really is trying to either
share that peer story or suggest a scenario which
they may find themselves in that is common, as opposed
to if we’re sticking to their caregiver or parent
or family or influencer, that’s where we go
to more of a statistic-based, so it’s a statistic-based message
instead of a situation-based message. But, again, all messages are
intended for consumption by those audiences. We just have the audience
in mind. We’re not able to separate
that in terms– I mean, all we can say
is we can compare the post that it’s intended
for are young adult and the post that it’s intended
for are influencer. These are the results for both. (Amanda)
Thank you so much, Brian and Thomas,
for your expertise and all the wonderful discussion,
and thank you everyone for taking the time to join
and take care, everyone.

About Ralph Robinson

Read All Posts By Ralph Robinson

Leave a Reply

Your email address will not be published. Required fields are marked *