Georgia State University was facing a growing “summer melt” problem, where nearly 20 percent of incoming students never actually enrolled. The university used a data-based approach to retain students of all racial, ethnic, and socioeconomic backgrounds and help them graduate.
Harvard Business School professor Mike Toffel and senior fellow Robin Mendelson discuss what the university learned about improving student success, while scaling its efforts to help other universities, in their case, “Student Success at Georgia State University.”
HBR Presents is a network of podcasts curated by HBR editors, bringing you the best business ideas from the leading minds in management. The views and opinions expressed are solely those of the authors and do not necessarily reflect the official policy or position of Harvard Business Review or its affiliates.
BRIAN KENNY: Today, learning happens in the classroom and in the cloud. There are approximately 5,300 colleges and universities in the United States enrolling about 20 million students. The National Center for Educational Statistics reports that about 60% of those students will stick it out to earn their degrees, opening the door to greater life opportunities, but what about the other 40%? Is there a way to help them stay the course? Today on Cold Call, we’ll discuss the case entitled, “Student Success at Georgia State University,” with case authors, Mike Toffel and Robin Mendelson. I’m your host, Brian Kenny, and you’re listening to Cold Call on the HBR Presents network.
Mike Toffel’s research examines company’s management of environmental affairs and occupational safety, identifying which types of management programs and regulations improve environmental and safety performance. Robin Mendelson is a senior fellow in the Harvard Advanced Leadership Initiative, but for 20 years before that, she held various leadership positions at Amazon, including leading the U.S. Books and Entertainment Media businesses. Thank you both for joining me today in our virtual studio here during COVID. Great to see you both.
MIKE TOFFEL: Thanks for having us.
ROBIN MENDELSON: Thanks Brian.
BRIAN KENNY: I think people will really enjoy hearing about the things that Georgia State is trying to do to address that 40% that I teased in the opening part of the podcast and I think it’ll be really relatable. I appreciate you being here to discuss the case that you both wrote together. So, Mike, I’m going to start with you. If you would tell us what would be your cold call to start this case when you walk into the classroom?
MIKE TOFFEL: When students arrive in the class, we teach this in our first year required Technology and Operations Management class or “RC TOM”, as we call it here at HBS. They have already had some exposure to process improvement through cases about Toyota and the Toyota production system or a train system in Japan. This case brings the idea of process improvement into an educational setting, a nonprofit setting, and one about large university, so quite different contexts. So I open by asking students which of the various tools that Georgia State has deployed to try and address this graduation rate issue most impressed them and why, and how would you figure out whether it’s effective? So there’s an evaluation lens that I bolt on right away. Then we take a few students who have different opinions and map it out on a board and off we go.
BRIAN KENNY: There’s so much to talk about as the case unfolds. Mike, you started to talk a little bit about how this connects to your area of scholarship, technology and operations management. Talk a little bit more about what inspired you to talk about Georgia State in that context, and Robin, I’d love to hear the same for you, but we’ll come to that in a second.
MIKE TOFFEL: So I was the course head for many years and thinking about how can we bring into the course process improvement in different contexts and contexts that our students might resonate with. They’ve all been to universities. They might think of themselves as possibly process improvement consultants in some manner and especially in this context of education where this particular university is serving a growing number of students who are first in their families to go to college, who are minorities and who are poor, coming from poor families. So it’s a really interesting, aspiring context for us to focus on and the school is trying to figure out how can we bring more experiences of the African-American students into our classrooms as well. So I credit Robin for actually bringing this whole space to my attention and she can talk about that.
BRIAN KENNY: Robin, I’d love to hear it. So obviously you’ve been at one of the best operational management companies in the world, working at Amazon for so long and have seen how they have managed their operations up close and personal. Was there a connection there for you, as you heard about what was happening at Georgia State? I’m curious as to how you became interested in this.
ROBIN MENDELSON: I learned about Georgia State’s story as I was researching low graduation rates of college students in the US, so much like what you were talking about in your intro. This has really been the area I’ve been pursuing as part the Harvard Advanced Leadership Initiative fellowship which incidentally, and it might be interesting to your listeners to know that this was founded by professor Rosabeth Moss Kanter of Harvard Business School really intended to stimulate new ventures and social impact work from business leaders. As you mentioned, my background, yes, it has been at Amazon. So I think one of the things that resonated so powerfully for me was having been so deeply steeped in this culture of customer focus and innovation using technology, testing and learning, using data to make decisions, and then this kind of a deep focus on operational excellence. It was that lens that I looked at Georgia State with and saw many elements in practice there and of course the facts of the case are so powerful. Here’s a university with thousands of students, low graduation rates, and it realized that some of its own processes were getting in the way of their students succeeding. So we can see that it launched all these initiatives and really improved graduation rates and eliminated equity gaps, but what was more interesting to me was really how they went about it, that they were really student facing, that they used continuous improvement. They were making database decisions. They were trying to use technology and other tools to make a difference in students’ lives. So that’s what affected me and really, I couldn’t stop thinking about it. I had an opportunity to share this story with Mike who was a business school classmate of mine many years ago, and it was really our conversation that led to the collaboration on this case.
BRIAN KENNY: Tell us a little bit more about Georgia State University. What’s the composition there, the makeup of the student body?
ROBIN MENDELSON: It’s a large public university. At the time of our case, it had about 25,000 undergraduate students and had grown materially in the last 15 years, about 30%. I think Georgia State’s growth was a direct result of the university’s intention to serve more students in Georgia. So during the time of the case, 70% of the undergraduates were students of color. Many, as Mike had mentioned, were the first in their families to attend college and about six out of 10 were from low-income families. So eligible for federal funding in the form of Pell Grants. So very different type of environment.
BRIAN KENNY: What were some of the challenges that they were facing?
ROBIN MENDELSON: I think that this case really focuses on student success. So it’s really within that lens and they had two significant challenges. I think, as you alluded to in your opener, the graduation rates were low. In fact, well below national averages. I think in about 2000 or 2003, it was about 30%. So seven of 10 starting weren’t getting through. During the course of the case, it was about four of every 10 were graduating with a degree at Georgia State. So that was one big area, and then the second is Georgia State really started to expand their aperture and accept more students, lowering GPA requirements and standardized test requirements in order to serve more students. They also had a problem with what’s known in higher education as summer melt. They accepted a lot of students and yet many students weren’t showing up at the first day of class. So those were their two large problems that we address in this case.
BRIAN KENNY: I was surprised as well to see that they count completion in a six year timeframe versus a four year timeframe. I’m curious about why that is, but I’m also, I guess, generally more curious about why the graduation rates were so low, Mike. I don’t know if that’s something that you guys looked closely at in doing the case.
MIKE TOFFEL: Yeah, sure. So the first question about the six years, there’s federal statistics on four-year graduation rates, five-year and six-year. They’re focusing on six year, in part because a lot of their student body are part-timers. A lot of them have to work during their college days and can’t take the time off to be a full-time student. The graduation rates were … Initially, they didn’t really know and they had to do a bunch of interviews and data collection to try and figure out why graduation rates were so low. Now, when we think about graduation rates, one shouldn’t think about it as a cliff where you admit a bunch of students and then they don’t graduate. Really, this is a problem of retention. So from semester to semester, and especially over the summer, they were losing thousands of students who didn’t come back. So they dug up some key problems with somewhere that academically students were failing out. They weren’t adequately prepared to take the classes that they were taking. Others were running out of funds. They had allocated a certain amount of funds either through their work or through student loans, and they just ran out of their eligible funding, and some didn’t feel supported. Again, students who are first time in their family, there’s a lot of unknowns about college experiences that families that have parents who went to college can coach their students about how to manage their own time, how to manage that flexibility. Those are sort of the high level reasons that they were assessing and they tried to dig in to try and figure out how can we address each of these problems.
BRIAN KENNY: Any parent out there who’s listening, that’s ever filled out the FAFSA form knows that you almost need a degree to be able to get through the financial aid process. It’s not an easy process to navigate and I can imagine for first-time students whose parents may or may not be there to help them go through this process, that it’s daunting to say the least.
MIKE TOFFEL: That’s right. Some of those forms require parental documentation, which some of these students might not have any access to. So that’s an easy example of some of the barriers to even figuring out how much financial aid you can even acquire.
BRIAN KENNY: Robin, I mentioned in the beginning that students who graduate have more life opportunities. I think that’s true, but maybe you can put a little bit more color on that by telling us what are the advantages for people who graduate and the disadvantages for those who don’t?
ROBIN MENDELSON: I’d say that graduation is very important both to earnings and to job creation. When we looked at this, census data shows that US workers with a bachelor’s degree on average earn 68% more than those with just a high school degree. So it’s very meaningful and in terms of lifetime earnings that amounts to about a million dollars of difference. So that’s significant. Then I think further to that with respect to job creation, Georgetown University Center on Education and the Workforce, which really studies this topic, estimated that by 2020, about 65% of jobs would require post-secondary education. It went further to show that 95% of the jobs that have been created in the recovery of the 2008 recession have gone to those with education beyond a high school diploma. So it certainly is worthy of focus on helping people get through their college education.
BRIAN KENNY: We haven’t talked about the protagonists in this case, Mike. Do you want to tell us who’s leading the charge on this effort at Georgia State?
MIKE TOFFEL: One of the things that makes this department interesting is that Tim Renick and Allison Calhoun-Brown, who are heading the student success group, which reports directly to the president, they were both professors, widely respected professors at Georgia State. It’s quite unusual to have an administrative group be filled with faculty at the head. So why is that helpful here? It’s helpful because they’re dealing not only with trying to review and revise administrative processes, figuring out for example, in the admissions process, what are the obstacles that need to be cleared away? How can we simplify things for students? But also they have to deal with faculty who are setting policies for, for example, what are the required sequence of courses? What’s the minimum grades that are required in order to enter your major? And really having faculty be the ones trying to ask those questions, I think goes a long way toward a receptive faculty audience. I think there’s always going to be some distrust between departments and a central administrative bureau, but having the leaders of that group be faculty themselves, at least I think gives them a bit of legitimacy.
BRIAN KENNY: They had a huge portfolio, the folks who are advising these students. Robin, tell us what the numbers broke down to be in terms of like one advisor’s responsible for how many students?
ROBIN MENDELSON: Their advising resources were really insufficient and many students never saw one. At the start, we had about 12 advisors for all of these students and the ratios were huge. One advisor to 1,200 students. So many students were just leaving, never, ever speaking to an advisor, which was one of the significant things they worked on in their process was to improve advising ratios and to use predictive analytics to help advisors proactively reach out to students and identify the students in the greatest need for contact.
MIKE TOFFEL: There’s an element of advising that’s really a data problem. Who do we target for our advice and what advice do we give them? That actually links back to some of my research which looks at auditors and inspectors because it’s the same question, like which entities do we target and what do we do once we arrive on site and how do we know that this is effective? So they were wondering, how can we figure out what’s the right body of students that we can intervene with in a timely enough manner to actually make a difference and that’s at the end, a data problem, and that’s how they treated it. It’s quite interesting, they basically built a machine learning regression algorithm that’s trying to predict how likely you basically to continue on the program or leave the program. It’s based on a whole range of factors, like: What were your high school grades? What were your grades last semester? Which courses are you taking? How many courses have you failed out? What’s your cumulative GPA? What’s your GPA trend? You put all of this into an algorithm to predict, how likely are you to leave over the summer versus come back next summer. That gave them a score from a zero to a hundred percent chance that you’re going to come back or you’re going to leave and from there, they categorized them into a red, yellow, green stoplight context and allowed them to design in a much more effective way, the different messages that they’d send to these different students.
BRIAN KENNY: I love that example and I would imagine that there are people at state universities across the country who would really, really benefit from hearing about this and not just … I think private universities too. We all have so much information about our students that it’s hard to make sense of it sometimes and it sounds like they really found a way to break that information down and to make sense of it and to make actionable sense of it so that they can actually intervene based on what they’re learning from the data. So Robin, again, with your Amazon background, this must have been really interesting to see the way that they were doing this.
ROBIN MENDELSON: It really resonated powerfully for me, not just the predictive analytics that was allowing these advisors to proactively and efficiently reach out to the right students who were at risk of failing, but what I really found was Georgia State took a sort of end-to-end ownership approach of the student’s experience. They started with the enrollment pipeline all the way through the academics to graduation and they looked at that student’s journey, standing in the student’s footsteps, trying to figure out where was the university responsible, where was the university creating friction? They tried to identify these large areas of friction for students, understand the root causes, use data and other things to measure the impact and then work away. I think the advising was one of many examples of how they did this, and I think that really resonated powerfully for me thinking about the way Amazon looks at a customer’s journey through the order pipeline and where did a customer abandon its cart, for example, and then looking at the data and doing root cause analysis and trying to figure out what are the reasons that a customer has gone away. Similarly, the student is really abandoning the student experience and dropping out. I thought it was actually a quite remarkable and the number of tools that Georgia State did employ including, as Mike has just mentioned, this kind of innovative use of predictive analytics and algorithms that was new in the university setting, but certainly is in place when thinking about how places like Amazon and others use those kinds of algorithms to help present products to customers and many other ways of doing this. So yes, it was really powerful to me.
MIKE TOFFEL: We talked about analytics. The other two major approaches that they used were process redesign and micro-funding, and there’s some really interesting examples there as well. On process redesign, for example, one of the things that I found to be quite interesting was nearly half of their undergraduates were failing their introductory math course. So there’s a variety of ways to try and figure out how to address that. You could raise your admission standards, which was not what they were trying to do. They were trying to serve as many students as possible. You could invest resources by making these classes a third of their size, but that’s very costly because you have to invest in three times the amount of instructors and you have to find classroom space. What they did was to basically implement what they call adaptive learning, which is basically flipping the classroom. So instead of having these large lectures, they had a lot of the student learning occur on an individualized basis through computer tutorials where, a bit like a Khan Academy, if you demonstrate mastery, then you move on to the next lesson. If you need more help, then it gives you more examples and more opportunities to dig in and have different types of video tutorials and so on. So that’s an example of really changing how the process worked and they did that in a number of ways.
ROBIN MENDELSON: I love the example that Georgia State shared with us about another process change they made that was a simple one and a brilliant one, too. They were finding that in the enrollment process, the state of Georgia was requiring students to provide their vaccination records as a prerequisite to enrolling. When they expanded this group of students that they were wanting to bring to Georgia State who had accepted the offer and said they were coming, and then they saw they were losing them in this stage of the process. So what they did was when students came for orientation, they brought in a mobile health unit, and if a student didn’t have their vaccination record, they were just offered an appointment to have that taken care of. You mentioned micro-funding and I think that doesn’t have as much resonance for me about my Amazon days, but certainly is a tool that is used in sub-Saharan Africa or in India, micro grants and micro-funding. It was another significant reason, as Mike has mentioned that students were dropping out. They had financial considerations and financial concerns. The Georgia State Student Success team looked at this data and they realized that each term they were losing about a thousand students. They were dropping them and it was another Georgia State requirement that if a student had an outstanding balance, they weren’t allowed to continue and they had to withdraw them. They had to drop them. So students were paying some of their balance, but they weren’t paying all of it, and Georgia State was, by necessity, pushing them out. When they started to look more deeply at those students, they realized that many of them were seniors. They were on track to graduate. They had strong GPA’s and except for this small outstanding balance, they would be on track to receive their degrees. So they did something interesting. They took a $40,000 gift from President Mark Becker, who was very supportive of all of this student success work and they used it as a pilot. They went and they looked at the seniors and they started paying off the balances. They gave them a micro grant and 34 students received these balances and about 70% of them graduated within the next two semesters. So it was a significant graduation rate and what they compared it to is about for all of those students that they were dropping, only about 30% of them were coming back. Then they expanded this program to about 2,600 students per term now received micro-grants and it seems to be a significant win for the university. They get the full tuition and they pay a little bit of a discount and then the students graduate at about 78% now within the next two terms. So it’s been very successful.
BRIAN KENNY: So I have three more questions before I let you go. So the first one would be, how’s it going, as we look back and then connected to that, for other universities that are listening, is this replicable? Can other schools do what Georgia State is doing, assuming it’s working. So let’s start with how’s it going. Robin, do you want to take that one?
ROBIN MENDELSON: So yes, we leave students, in this case, with the problem that Georgia State Student Success team has just realized they had. They’ve now increased the number of students being admitted and this is overwhelming their admissions officers. They don’t have the capacity to respond to all these students. So what was happening was their summer melt. This percentage of students who say they’re going to be joining Georgia State, they accept the offer, they don’t actually show up. It was about 20%. So they were losing 800 students who they expected to start the year. Well, some of the students were going to Georgia Tech or other universities. So that seemed to be a good thing. Some students were staying closer to home and going to community colleges, but about 300 students were going nowhere and Georgia State realized that these were the students that they most wanted to serve. Students coming from low-income families. They were first-generation students, they were students of color. So this is the problem that we leave students with at the end of this case and it shows the Georgia State team starting to explore some solutions.
ROBIN MENDELSON: One of them is evaluating the 14 steps and building a process map of all the steps that students go through. Again, looking for areas of friction, looking at the end-to-end student process. The other one was using an innovative technology-based solution is to think about a bi-directional chatbot texting, using artificial intelligence and connecting it to the student information system might actually be a way for students to have some of their questions answered. That’s what we leave students thinking about at the end of the case.
BRIAN KENNY: The chatbot, that’s an interesting tool and we’ve all experienced, I think, a little bit. If you’ve called Verizon or your local utility, you’ve probably spoken to a chatbot. Mike, how do you think it’s going and do you think it’s replicable in other places?
MIKE TOFFEL: Well, the case also describes how to make it replicable, and it lays out two initiatives that were being contemplated. One is this alliance with other large public universities who face similar challenges at scale and how should they engage with that group productively. The other is that Georgia State itself after hosting so many tours to folks who are interested to learn their innovative practices like we were, they were thinking, well, is there a better way to do this than just hosting tour after tour saying some of the same things? So they were deciding about launching an educational program on how to do this so that they can launch the next generation to go deploy this approach and these tools at universities around the world. So those are two mechanisms we asked students to evaluate on the scalability question.
BRIAN KENNY: What about on the graduation side? How are they doing with graduation rates?
ROBIN MENDELSON: Well, I’m glad you asked. I think the story is really inspiring and Georgia State around the time of the end of the case was at about 54% graduation rate. Now I think it’s closer to 57%, but really what was so inspiring was that the work that they have done over these years has made such a difference for all the populations they serve. So now Georgia State’s African-American, Latinx, and low-income students all graduate at, or above the mean, the average of Georgia State, which was not the case early on. In addition, Georgia State really is known for the last several years, since I think 2001, of being the largest conferrer of bachelor’s degrees for African-American students. So it really has made a significant difference.
BRIAN KENNY: This has been a great conversation, really so interesting to hear what they’re doing. So I have one question for each of you before we end our session, and that would be, if there’s one thing you want people to remember about this case, one thing you’d like them to take away, what would it be? So, Robin, let me ask you to start.
ROBIN MENDELSON: I’d say that the piece that was really moving for me is to think about education as a lever that can really change the trendlines in a student’s or an individual’s life, increasing their financial and social mobility. So what I want people and future business leaders to think about is to see that many of these transformative change fundamentals that are meaningful at other companies and other organizations, when applied at scale can really also lead to significant social impact as they have for Georgia State students.
BRIAN KENNY: Mike, your thoughts,
MIKE TOFFEL: Similarly, actually. I was interested in writing this case with Robin to bring to our first-year operation students and to hopefully students around the world who are exposed to the case, the idea that process improvement is not just the Toyota production system in factories. It’s not just about culture change and bringing ideas from frontline workers to management’s attention, but also these tools of process redesign and simplification, the idea of data analytics, which people are familiar with often in platform contexts like Amazon, or like Uber, can be deployed surprising contexts like nonprofit service offerings, like a university.
BRIAN KENNY: The case is, Student Success at Georgia State University, a great case. Thank you both for writing it and for joining me today. Great to have you on the show.
ROBIN MENDELSON: Thanks for having us.
MIKE TOFFEL: Thanks so much.
BRIAN KENNY: If you enjoy Cold Call, you should check out our other podcasts from Harvard Business School, including After Hours, Skydeck, and Managing the Future of Work. Find them on Apple Podcasts or wherever you listen. Thanks again for joining us. I’m your host Brian Kenny, and you’ve been listening to Cold Call, an official podcast of Harvard Business School brought to you by the HBR Presents Network.