Cheryl Broom [00:00:04]:
Hi, I'm Cheryl Broom, CEO of GradComm and host of Higher Ed Conversations sponsored by
EdTech Connect. Today, we have a great guest on something we haven't really dove into before
which is predictive modeling and how colleges can use it to address their biggest challenges in
real time. Emily Coleman is the co-founder and CEO of HAI Analytics. And throughout her
career she's worked with higher ed institutions to predict and improve yield, effectively allocate
financial aid dollars, and ultimately support student success. Emily began her career in
enrollment management at Syracuse University, where she worked for 12 years, most recently
as the assistant vice president of enrollment management. While she was there, she built
modeling strategies that fueled record applications and enrollments. She also served as senior
vice president in enrollment services at McGuire Associates. And while in this, she led the
company's predictive modeling division and was a member of the senior leadership team.
Cheryl Broom [00:01:08]:
Today, Emily is the co founder and CEO of HAI Analytics and they work with colleges and
universities all across the United States really to help them solve all sorts of problems. I saw so
much synergy between the type of work that Emily and HAI analytics does and what marketers
want to be doing at community colleges and universities. I think you're going to be full of ideas
and great takeaways after this conversation. So let's get started. Emily, thank you so much for
joining me today. I'm so happy to have you here.
Emily Coleman [00:01:46]:
Thanks for having me. I appreciate it.
Cheryl Broom [00:01:48]:
You have a really fascinating business. So I want to start off talking just about HAI Analytics and
what you do.
Emily Coleman [00:01:56]:
Sure. We are a full service higher education consulting company. What we do the most of is
building predictive models. So statistical models that predict student behavior from inquiry to
applicant conversion, admit to enroll, yield and then retention. We sort of build predictive models
all along that student life cycle. And we help colleges and universities spend their financial aid
dollars efficiently to meet their goals. So, you know, whatever. If they have certain programs that
they have to fill or certain areas where they're challenged, we kind of take a look at all the
factors that influence yield and come up with financial aid strategies based on that.
Emily Coleman [00:02:42]:
So that's the bulk of what we do.
Cheryl Broom [00:02:44]:
Now to take a step backwards for those people who are like really new to this, what exactly is a
predictive model?Emily Coleman [00:02:52]:
So a predictive model is basically just the mathematical equation. So we use. There are different
statistical tests that you can run. Now everyone's sort of heard of AI and machine learning
models. Those are the newest versions. There's older versions that are still quite reliable. So
we're building a regression model, and we're looking at all of the factors basically that we can
include. So for, you know, a yield model, we're looking at everything that comes in on the
application, you know, demographics and GPAs and interests, and then also everything that
comes in on the financial aid application.
Emily Coleman [00:03:31]:
So we're looking at whether the family has demonstrated need, did they apply for financial aid,
and what offer did they get. And it just, you know, it sort of comes up with a. An equation that for
a human to do it on paper would probably take a year, but it takes a computer, like, you know, a
matter of seconds to do it. And then once you have that equation, you can vary factors and test
what the outcome would be. How would the outcome be different based on factors that you've
tested?
Cheryl Broom [00:04:02]:
So it's actually interesting that this podcast is today because speaking of predictive modeling,
my older son is on a competitive soccer team, and one of the moms found an AI predictive
model of whether or not they'd win the games and by how many. So we had to actually stop
sharing it with the kids because they were getting so worked up over the predictions of the
game. But I'll tell you, this score has not been correct. But of the last eight games, whether or
not they win or lose has been 100% correct each time.
Emily Coleman [00:04:34]:
So, yeah, yeah, if you have the right data, you can basically predict anything.
Cheryl Broom [00:04:39]:
Yeah. So for your predictive modeling, what are you looking at? Whether or not a student will be
successful, whether or not they'll continue? What are some of those points that you're
examining?
Emily Coleman [00:04:50]:
Yeah, we're looking at all of that. So when we build a yield model and admit to enroll. Yield
model, you know, we're looking at kind of shaping the class, what types of students will enroll.
But then we're also looking at who will be successful, who will be able to continue and graduate
from the university. Of course, that's the end goal. So we sort of see retention as an enrollment
goal. We want to know students likelihood, and really we can build models that predict retention
from only information we have before the student arrives on campus. So just using kind of the
application and the financial aid information, we can identify students who might be more at risk
of dropping out, who might need some extra support.
Emily Coleman [00:05:36]:And we can sort of assign likelihood scores based on, you know, your likelihood to retain your
likelihood to enroll. So schools can really use that to build support services around students
that, you know, are helpful to everybody.
Cheryl Broom [00:05:49]:
Well, I love this for community colleges because there's not unlimited resources so having some
data to show students who might need extra support from the beginning, particularly
communication support, would be invaluable. So you're not wasting time talking to 18,000
students. Perhaps now you've identified 1800 and can really reach out and start supporting
them.
Emily Coleman [00:06:13]:
Yeah, and that's exactly what kind of the inquiry to applicant models that we build we're dealing
with. You know, some schools have inquiry pools of 200,000 students and we can, you know, as
you said, narrow it down to like 20,000, have some chance of applying. The rest of these are not
going to apply and they're definitely not going to enroll so that they can direct, you know,
expensive publications or outreach to those students who have a higher likelihood. And it's just
a more efficient process.
Cheryl Broom [00:06:43]:
Wow, that's interesting. You know, so we don't do a lot of work with four year universities. So I
was just thinking, like, wow, not everybody gets one of those fancy. Like not all 200,000 people
interested in UC Santa Barbara get one of those.
Emily Coleman [00:06:57]:
Right, Right. Yes, definitely not. Yeah.
Cheryl Broom [00:07:00]:
Fascinating. One of the things that we see with our clients is that their systems just don't talk to
each other. So here in California, for example, all 115 community colleges share a common
application. That application is run by the state chancellor's office. But then they may have a
homegrown enrollment system. So the application you can't track the student through. Is that
something that your company can assist with? And how do you, how do you do that when you
have all these different systems that don't talk to each other?
Emily Coleman [00:07:33]:
Yeah, so we use a software product called Alteryx Designer and it allows for disparate data sets
to be kind of merged together. It has a lot of, you know, tools that if you don't. Sometimes we
find that schools can't even exactly match the data sets. So they may have one type of ID in one
system and another type of ID in another. So we can do things like fuzzy matching, which sort of
pulls all of the things that are, you know, the same and matches students that way. And we can
put that together into a workflow that a college or school can use to, you know, run it to get a
data set that includes everything from every system and use that, of course, to make strategic
decisions, because as you said, if the systems don't talk to each other, it's very difficult to dothat. So, yeah, we have worked with a lot of schools where that's the issue and we can kind of
help them with the merging of those files.
Cheryl Broom [00:08:35]:
Great. And what Type of things are you looking for? I mean, is this all AI generated or is there a
human component to it as well? What are the data points that you're searching for?
Emily Coleman [00:08:46]:
So you know, we're, we're looking for likelihood of a student engaging in a certain behavior. And
so with statistical models they will predict kind of or they'll create a predicted outcome score for
each student. That's kind of the AI side. And you know, there's a lot of companies now talking
about we use AI models to predict your students behavior and they're highly accurate. And
that's true. But the human component is very important as well. So HAI stands for human and
artificial intelligence. And what my business partner and I saw in our previous roles is that
students come out of undergrad with incredible experience building statistical models.
Emily Coleman [00:09:33]:
They can build very complicated AI models, but they haven't actually implemented them in a
real world setting. No model is perfect because outside factors can change that the model can't
take into account. So a big part of what we do is interpreting the models and sometimes
adjusting them. If we have a school where we know the models always under predict
enrollment, then we can adjust that, we can account for that. So the human component is very
important. And I would say that machine learning and AI, they're buzzwords now, but they
definitely need the they someone with experience in our case in enrollment management to
know which parts of this do I trust, which parts do I question and kind of interpret it in that way.
Cheryl Broom [00:10:24]:
Well, I'd love to hear maybe an example of a college who has used your services to make some
changes. But before we get to real world example, let's take a quick break and hear a little bit
from our sponsor.
Unknown [00:10:37]:
How do higher education decision makers find the right solution when technology evolves at
light speed? Well, we usually start with our network. EdTechConnect is the network that's
democratizing the higher ed technology conversation. EdTechConnect is free, so anyone with a
edu email address can sign up and list the software and services they use in their role at their
school.
Cheryl Broom [00:11:02]:
Once you're in, you can find out.
Unknown [00:11:04]:
What solutions similar schools are doing all over the country. Whether you're looking to find a
hot new AI tool or maybe learn options, you have to upgrade your campus search.Cheryl Broom [00:11:13]:
Engine or even get to your short.
Unknown [00:11:15]:
Shortlist of marketing solution vendors. EdTechConnect is the place to go. So visit
EdTechConnect.com and set up your free profile to get a pulse for what's happening with higher
ed technology.
Cheryl Broom [00:11:27]:
Today. All right, so before the break, we were talking about. We were talking about human and
artificial intelligence, which I didn't know that's what your company name stood for. And I wanted
to see how you've actually applied this. So share with us an example. And you don't have to say
names because I know some of your clients are probably pretending with and are confidential.
But just how. How has a college, a community college or university used your services to make
improvements?
Emily Coleman [00:11:57]:
So a lot of our, you know, all of our clients have specific enrollment goals. They have headcount
goals. Of course, they have revenue goals. And the models help develop a strategy that, you
know, in unison with their expertise and the work that they're doing on the ground gets them to
those goals that they're, you know, that they're trying to achieve. But I think one of the most
impactful things that we do is looking at first to second year retention before the student arrives
on campus. So looking at when we develop a financial aid strategy that maybe in the first year it
meets all, you know, checks, all the boxes. It's the revenue, it's the headcount discount rate. But
in year two, that may, you know, something that's producing more revenue in year one means
students are getting less aid, they may be less likely to retain.
Emily Coleman [00:12:50]:
So if we look at the revenue over two years using this retention model, sometimes we find that a
strategy that's more generous in year one will net more over two years than a strategy that's
less generous. So I think that's a, you know, that's something that it affects the institution. It
helps them to operate effectively. And also it's good for the student that, you know, they're sort
of looking out for that and they're not just trying to get you in the door. They really want you to
be successful and be happy alums someday.
Cheryl Broom [00:13:21]:
Yeah, that's great. And I love the idea of using this for retention because one of the problems we
see and that we help a lot of our clients with marketing wise is Summer Melt, where we've got
students who've completed a year and then all of a sudden they disappear. And knowing if there
is a certain type of student that's more at risk for not returning would really help. It would help
marketing tremendously. Because right now we're just doing like broadcast campaigns like,
don't forget to come back. We want you to come back. But nothing like supportive or
personalized. So I could really see a big use for that.Emily Coleman [00:14:00]:
Yeah, yeah. And you can even look, I mean, Once you have kind of a model that say, predicting
retention and you know, what factors are, you know, statistically significant predictors at that
institution, you can not only identify who's at risk, but you can sort of identify why they're at risk.
So, you know, is this person at risk because they're from a far away state and they traveled far
to get here? Are they at risk because they don't have enough financial aid? You know, what,
what is it about them that is making the model tag them as at risk? So then the school knows
not just to respond to them, but how they should do it.
Cheryl Broom [00:14:37]:
Yeah, I see like a lot of applicability too for our applied non enrolled students. So our students
who applied and completed orientation maybe are more likely to actually register for a class. So
now we know we got to get them through the orientation. Like that should be our first
communication touch point is get orientation completed because we know they're more likely to
register. I think that's data that most of our colleges that we work with just don't look at or
consider.
Emily Coleman [00:15:08]:
Yeah, yeah. I mean, in the community college system there are so many, you know, things that
they're worrying about and they, they can tend to be very big and as you said, kind of large
systems that have people at different institutions behaving differently. So it can be more
complicated. But certainly if the data are there, even if they're not in perfect shape or they're not
married together, then we can definitely build some data driven strategies.
Cheryl Broom [00:15:41]:
And even here in California, I've got my head, my head is spinning now with ideas. You get a
greater apportionment, for example, for first generation students. So if you have more first
generation students, you're going to get a higher dollar amount from the state than you would
for a student whose parent had gone to college. So, you know, most community college
students are first generation students. So I mean, now you can even look at that cohort and say,
like, historically, how have our first generation students, like, what do they have in common,
those that succeed? Have they gotten scholarships? Have they gotten more financial aid? Do
they go part time? Do they go full time? So it would be so fascinating to look at your own
institutional data and do some predictions around students that will enroll and ultimately
succeed. Yeah, really interesting. One of the things that our colleges are struggling with too is
just enrollment in general. And are you able to use your modeling to predict enrollment
increases or decreases? Is that something you could help with?
Emily Coleman [00:16:48]:
Yeah. So the models are really highly accurate at predicting what the yield of a pool will be how
many enrolling students there will be and whether a school can anticipate increases or
decreases based on even sort of the very top of their funnel, like the number of inquiries they
have or where they're from. We can kind of follow that all the way through to be able to predictchanges. And you know, typically what we're doing is where creating strategies to, you know, in
spite of those changes to hit the goals. So if, if we expect this pool has a lower likelihood of
enrolling based on, you know, a whole, a whole collection of factors, they may need more
financial aid or more assistance to enroll. So that's, we're kind of looking at it and, and coming
up with a strategy that will help deal with that situation once it does come.
Cheryl Broom [00:17:43]:
Yeah, I love, I just love this data driven approach because what we see with our clients is there
hasn't been any predictions, there hasn't been any data look or historical analysis. And now it's
June and enrollment's down and marketing who's like a one person shop all of a sudden
everybody's freaked out and they give them a bunch of money and they expect them to find
students to replace what they've lost in the next two months. And our company comes in and
tries to help them do that to the best of our knowledge. But you know, getting a bunch of new
students is, is a way harder than knowing why you lost all these students to begin with and
trying to fix that problem.
Emily Coleman [00:18:28]:
So. Yeah. And I mean that's, you know, my favorite kind of client schools that have never done
this before because you can see such big differences when you start to implement these tools.
And so it's really kind of fun to see those impacts when they're kind of that big.
Cheryl Broom [00:18:48]:
Yeah. And also just to see like we don't have a marketing problem, like an X, Y and Z problem
or maybe this type of student needs this type of, more of this type of support. So I think that type
of intelligent data is going to help colleges use their resources better.
Emily Coleman [00:19:05]:
Yep, yep. And that's what it's all about.
Cheryl Broom [00:19:07]:
Anything else we haven't talked about that your models can be useful for?
Emily Coleman [00:19:12]:
You know, they can be useful. We have done models like with law schools. We've built models
to predict bar passage.
Cheryl Broom [00:19:21]:
Oh wow.
Emily Coleman [00:19:22]:
And we've been able to kind of tag the students who are most at risk of not passing the bar. And
we've had law schools use our findings to sort of shape the curriculum and shape support
services services. So, you know, the findings really are applicable in Lots of different settings.
We've worked with medical schools, We've worked with kind of all kinds of schools andcolleges. And so being able to predict that student behavior is, you know, something that
everybody needs to do, and it's applicable to every type of institution. So that's mainly, you
know, what we do.
Cheryl Broom [00:19:58]:
Yeah. And what do you have some challenges, like if you start working with a college or
university, what are some things that are roadblocks for you to do your work?
Emily Coleman [00:20:07]:
So one big roadblock, although this isn't as true anymore, is schools purging their financial aid
offers to students who don't enroll. So if we want to study the effect of financial aid dollars on
enrollment, we have to have those offers for both enrolling and non enrolling. And particularly, I
would say, like 10 years ago, it was very common to see that schools have purged those data.
And you know that that is a huge setback if you're trying to do yield modeling. So that's one of
the big ones that we face. I think another pretty common one is that an institution is kind of
siloed. And so we're hired by the admissions folks, but the admissions folks and the financial aid
folks don't really talk to each other. And that is, again, something that I feel like has gotten better
over the past a few years.
Emily Coleman [00:21:01]:
But that is something that can pose a challenge for sure.
Cheryl Broom [00:21:05]:
Oh, my gosh. Silos is the name of the game when it comes to community colleges.
Emily Coleman [00:21:12]:
Yeah, yeah. Well, higher ed in general, they love their silos.
Cheryl Broom [00:21:15]:
Yes, well, and everybody is so busy too. I mean, they're just trying to keep from drowning, and
they don't have time to go and work with other departments. And then that creates this, like,
perpetuating cycle of just, you know, not working with each other.
Emily Coleman [00:21:30]:
So. Yep, yep.
Cheryl Broom [00:21:33]:
I just. For those listening, referring Emily to a. To a college whose president wants to see where
students are dropping off in between application and enrollment. And she actually came as
president, came to me and said, can't you just build a spreadsheet? And I was like, you have
literally like 10 different systems going. Like, I don't even know where to start. And I was like, oh,
I just met somebody who might be able to help with this. So I'm really hoping that they'll connect
and we can get some. A really good use case scenario for one of our California community
colleges.Emily Coleman [00:22:04]:
So, yeah, yeah, that would be great.
Cheryl Broom [00:22:06]:
Fingers crossed. But yeah, when I started digging, I'm like, so you've got an application system,
an enrollment system, an orientation System counselors are entering things into different
systems. There's this inquiry form that nobody even knows where it goes to. It's not being
entered anywhere. So many different things bringing it all together. Well, as we wrap up our
conversation, do you have any advice, maybe for cars? Colleges who aren't ready to hire
somebody like you to come in, but might want to just start thinking about how they can better
use their data. Is there any advice you can give them?
Emily Coleman [00:22:40]:
I think one big piece of advice which I sort of just alluded to is do not purge any of your data.
You know, schools kind of, it's not expensive to keep data now, and oftentimes the financial aid
data are being purged because they don't want to accidentally send an award to someone who
didn't enroll. But if you want to do anything, you know, any data driven strategy, you've got to
have as much data as possible. So don't get rid of any of it. And then, you know, they can. I
mean, we'll go into an institution and sort of do that data audit before we sign a contract with a
school. Just to say, here are some areas where you really need to change your process or you
need to start recording the data. And then once you have a cycle or two like that, then we can
come back and build the predictive models.
Emily Coleman [00:23:29]:
But it is important to start thinking about how do we get to that point where we could build
predictive models. And I think a lot of schools kind of, or just people kind of get. They freeze
when they realize, like, we have so far to go to be able to predict those models. We don't. We're
not recording the data in the right way. We're not keeping the data. And it's hard to think about
like moving to the next level because as you said, everybody's busy. But it definitely can be
done and we can help.
Emily Coleman [00:24:01]:
And I think after sort of the initial paying period, everything gets easier because it's easier to
make decisions.
Cheryl Broom [00:24:09]:
Well, and I was going to ask you how people get started, and it sounds like that's a great way to
get started is just to have an audit.
Emily Coleman [00:24:16]:
Yep.
Cheryl Broom [00:24:17]:And once this model is built, is this something that UC colleges are able to maintain on their
own or do they continue to use someone like you to come in annually and help them?
Emily Coleman [00:24:27]:
Most of them continue to use us. One of the services that we provide is that if they want to bring
it in house, we'll help them get there. So, you know, if they have someone who can build
statistical models but has never done it in this kind of area, we can work with them to show them
this is what you need to look for. We've done some of these where it's been a three year
agreement where the first year we do all the model building but that their person kind of
shadows us and we explain what we're doing and why we're doing it. In the second year, we
sort of run parallel processes and we're evaluating the way that they're doing things and they're
looking at the way we're doing it. And then in the third year, it's kind of the least involvement on
our part where they've taken over the modeling and we're just kind of looking everything over to
see if we see any red flags or any issues and then they can continue and kind of do it on their
own. And a lot of people said to us, that's a terrible business model. You're teaching your clients
not to need you.
Emily Coleman [00:25:31]:
But there are a lot of institutions in the world, so we feel like just helping schools become more
efficient in whatever way is something that we, we really take joy in.
Cheryl Broom [00:25:42]:
Yeah. And we're the same in our company. We, I tell, I always tell our clients, if you get to the
point where you can do this on your own, I think it's great.
Emily Coleman [00:25:50]:
Yeah, yeah.
Cheryl Broom [00:25:52]:
Get there. Well, Gray, I, I think this is so exciting. I just see for, for the marketing perspective
how having information to this, how having access to this information in this data can just make
your marketing so much more targeted and more effective, more cost effective and better in so
many ways. So I'm really excited, I'm excited to see if we can get some community ecologists to
start using.
Emily Coleman [00:26:18]:
That would be great.
Cheryl Broom [00:26:20]:
So I want to thank you, Emily, so much for being on the podcast. And if people do want to get in
touch with you or connect, how can they do that?
Emily Coleman [00:26:28]:They can go to our website, haianalytics.com and there are forms on there that they can submit
with their contact info and we can get back to them. They can also find me on LinkedIn and
reach out to me that way.
Cheryl Broom [00:26:40]:
Wonderful. Well, thank you so much and I've learned a lot. Really appreciate it.
Emily Coleman [00:26:45]:
Thanks for having me.
Cheryl Broom [00:26:47]:
And that wraps up this episode of the Higher Education Conversations podcast. I'm host and
GradComm CEO Cheryl Broom. A big thank you to our sponsor, EdTechConnect.
EdTechConnect is free, so anyone with a EDU email address can sign up and list the software
and services they use in their role at their school. So visit edtechconnect.com and set up your
free profile to get a pulse for what's happening with higher ed technology today. And while
you're online, take a few minutes to leave our podcast a five star review. It will help other
colleges and universities find us and learn from the great experts we have on the show. That's it
for now.
Cheryl Broom [00:27:27]:
Until next time.