.png)
Listen to this article
What does MLOps look like when you are deploying 22,000 models a month?
Maddie Daianu, Head of Data and AI at Credit Karma, joins the Data Bros to pull back the curtain on one of the most high-volume data environments in FinTech. With a 100-person team serving 140 million members, standard data practices break down.
Maddie shares how her team manages terabytes of daily data on Google Cloud and explains the massive strategic pivot they are undertaking right now: The move from "Information" to "Agency."
Listen on Spotify or Apple Podcasts
[00:00:01] Benjamin: Alright. Hello, everyone, and welcome back to the Data Engineering Show. Today, we're super happy to have Maddie on. Maddie kind of works at Credit Karma. She leads data and AI there. Super great to have you, on the show. Do you wanna quickly introduce yourself, kind of give us some background on what you're working on, yeah, and how you got into data?
[00:00:23] Maddie: Of course. Thank you so much for having me. It's a pleasure to be here today. Yes. I am Maddie, and I lead data and AI at Credit Karma Intuit. I've been at the company for about two and a half years. Here, the team for data and AI consists of AI scientists, machine learning engineers, data practitioners, including data engineers, assist intelligence engineers, and the experimentation platform. And this team has been at the genesis of how we engage our members across Credit Karma, which is essentially a financial app that helps you not only advance your credit score but also find a marketplace of very useful and relevant highly personalized offers for credit cards, personal loans, or anything that you might be shopping for. And, it's enabling to you to advance that financial journey that you're on. So for that, the key elements and ingredients of making this app successful is data and ai. So I'm very very excited to like be telling you a bit more about how we build the team and really what are some of the competitive advantages but also challenges that that we are seeing. a bit of background about me before being at Credit Karma, I was at smaller companies, but also at Meta at some point. And then my first job out of academia was at Intuit. I kind of came back full circle. Before Intuit, I was an academic working on biomedical engineering, using at the time machine learning to extract biomarkers of disease for Alzheimer's. I really wanted to essentially hopefully see if I can change the path of alzheimer's in my lifetime. I learned that that might not be the case, so I switched into industry where you can see a faster pace of innovation. ultimately though what I'm grounded by is a very consumer facing mission. So I love helping users like yourselves um see the benefit of technology.
[00:02:28] Benjamin: That's amazing and like what a wide range of things, seriously. I mean like going kind of from biotech, kind of smaller companies, bigger companies. Very, very cool. So maybe before we dive into the data and tech stack and kind of your teams, we have a lot of listeners from kind of across the globe. I'm sure many of our US listeners or virtually everyone, will know about Credit Karma and also Intuit. Do we maybe wanna give, like, a one minute kind of background for every kind of listener who has not heard of these companies in the past? kind of, yeah, what they're all about, kind of what what your products enable.
[00:03:03] Maddie: Yeah. Absolutely. So maybe starting with Credit Karma, which is, again, a financial product that enables you to check your credit score for free. And that was the genesis of Credit Karma from maybe more than ten years ago, and our founders really wanted to lean in to help you find your your way to advancing your your financial progress and making this easier without having to pay for information that can be readily available at your fingertips. Now, of course, credit scores, you can find them on a wide range of apps. So we've evolved that journey of our products to fully personalizing with highly relevant machine learning models that enable you to make the right decisions given what you need in your financial journey. Say that you might want to change or evolve your credit score, you might need very different types of support either consolidating your debt or finding ways to save money that are different from people who might have a very different financial goal. So that's credit card bond. Now we are part of intuit. So the beauty of of uh this uh essentially journey that we are on is that we can combine credit scores from credit karma with people who want to do their taxes on turbo tax. So turbo tax is also a very successful product that has been providing a lot of customers across the globe, but especially in The US, the ability to do your taxes also in a very personalized way. So think of it this way, credit card is almost a front door to turbo tax, and one thing that we are doing uh at intuit right now as part of what we call this consumer ecosystem is building synergies and personalization and seamlessness across credit karma and turbo tax such that we can fully understand you as a user as you traverse these products. And that is really what's behind what I believe is one of our biggest competitive advantages as a company.
[00:04:56] Benjamin: Awesome. Cool. So let's let's start chatting about the data stack. Right? Just kind of like, how big are your teams? How are maybe they structured? What types of data do you bring in? Kind of what pieces of software do you have in your tech stack? We'd love to learn more.
[00:05:12] Maddie: Yeah. Absolutely. So my team specifically is about a 100 people, but we do have, over 800 engineers that are part of Credit Karma. So it's very, very engineering heavy. and we essentially build a, engineering first mindset from the genesis of the company to how we operate our teams. Specifically for my team, as mentioned, we have AI scientists, which are about a third of my team, and then we have machine learning, engineers as well as data experts who build a platform that enables us to run, the training for machine learning, but also facilitate the deployment of those. And then I have a third of the team that is more data practitioners for business intelligence as well as the experimentation platform. Credit Karma is very specifically a Google Cloud, infrastructure. They transitioned to this a while back. So think of it this way, BigQuery is our data warehouse, we use Bigtable for our operational serving layer, and we ingest large information from the financial institutions that we work with, from our partners that we work with every single day. so we have and we process and transform multiple terabytes of information daily for our 140,000,000 members every single day. So like that's a very exciting large data set to like have our hands on and be able to, provide personalized information for. So Yeah.
[00:06:40] Eldad: Of course The only problem is BigQuery. The only problem here, like, everything is perfect. The stack is amazing. Everyone, like, participants, everything. But just BigQuery, come on. We need more. More spice. Can't have BigQuery solve all problems. Just kidding. Go on.
[00:06:57] Maddie: It's not. It's it's certainly not solving all problems. It's something that we have learned to essentially, ensure we build holistically for for the company. Nonetheless, we are, very much focused on using Google as as an infrastructure. Having said that, we also are focused on multi cloud integrations because Intuit is an AWS shop. So we are essentially, when you think about the TurboTax products, but also like a wider range of products that Intuit has, they are in AWS. So we have to be oftentimes cloud agnostic to be able to facilitate, the integration of our cloud products and serve our members effectively across, end to end consumer ecosystems. Now
[00:07:43] Benjamin: And then looking like, one thing I'm curious about looking at kind of your utilization of BigQuery is most of your analytics, like, internal and, like, kind of batch based. So let's say every six hours, you refresh kind of credit scores and do some predictions, and then you expose it back to customer. Or there's also that kind of customer facing, almost like real time analytics challenge, where if I'm a customer of Credit Karma or kind of other Intuit products, I log on to the app, kind of go on to the website, and can kind of consume my data kind of directly in a live way with, like, kind of customized like, kind of take us through that internal versus external part of your data, data stack.
[00:08:25] Maddie: Yeah. So it's both. We have both streaming and batch pipelines depending on, the financial institutions that we serve or depending on the data sources. The patterns are are different. Let's say that you as a user log into Credit Karma and your credit score has changed. As soon as we learn that the credit score has changed, we update that in as freshly as possible. So you can see the most recent information within the app versus waiting for, say, a batch pipeline that might take weeks or months to update. So when it comes to that, we are trying to be very, timely. We're providing you information that is relevant. Similarly, if our partners change, say, certain aspects about their offerings and APR changes or similar, we also update that information as quickly as we learn so that users, again, get access to the latest, more import most important information to make the best decision for themselves.
[00:09:19] Benjamin: And then that last mile customer facing journey is also done on Big Query, or you basically start exporting into some other systems at that point?
[00:09:29] Maddie: So for instance, one of the Bigtable is where we use, as our operational serving layer. And also, we have something called, Alchemy, which is our feature online feature store, which also uses, big BigQuery and Vertex AI so that we can do transformations in real time and aggregations of information, and enable you to, like, see, derivations of the data, in real time within the app. And more importantly, what we use this for is ultimately fueling our machine learning models. So on top of our data, which I believe is not nearly as powerful if you don't really have that AI intelligence layer, we have our models that essentially, lead to almost 80,000,000,000 daily predictions for our 140,000,000 member base every single day. We have more than 22,000 model deploys every single month, so we can refresh and personalize the models so that we can meet our members where they're at. So for that also we use Alchemy, which is again our feature store to be able, to personalize that experience.
[00:10:38] Benjamin: Okay. Super super impressive scaling and a very, very nice stack. If you I'm curious about the ML side of things, because I assume that kind of here, like, what types of obviously, I'm sure you can't go into all the details, but, like, what types of models do you actually kind of train? Right? Because especially
[00:10:55] Eldad: in the
[00:10:55] Benjamin: finance space, things like it's, like, heavily regulated, of course, kind of there's, kind of explainability, fairness of models, and all of that really matters. so it would be super interesting to learn about some of your experience there.
[00:11:09] Maddie: Absolutely. So let me break that down into ways. One is, like, the machine learning stack, which is our bread and butter and tradition that would be traditional AI. And then we can talk about gen AI too, which I think is is very relevant and also important to discuss. So for machine learning and traditional AI, our bread and butter is recommendation models. And that includes essentially ranking, and personalization of, say, an offer either within the app or a marketing channel. So we have hundreds of those models that enable us to ultimately deliver the right experience for our users. And within those, of course, we have to be very cognizant of information that we can and cannot show based on compliance as we are in a highly regulated space. Now having said that, where compliance, for instance, especially as it pertains to our partners, comes into play is when it comes to gen ai, which, unlike traditional ai is non deterministic. Right? So oftentimes what we do and we've seen a lot of success with especially at product karma is using traditional ml and pairing that with GenAI for contextualization explainability purposes. So when we show you an offer for, say, a credit card or a loan, we have something called c y, which explains why are we showing you this particular offer such that we can build trust with the user but also educate on whether or not they should consider opening or taking that particular offer. Now when it comes to essentially say features like cy that contains more context about the offer, we have to be very thoughtful about the benefits that oftentimes pertain to our partners that they are represented with a 100% accuracy. So that's where we invest tremendous effort into evaluation platforms so that we make sure that everything is a 100% accurate before we, roll it out to users.
[00:13:02] Benjamin: Awesome. Super interesting. And then if if you look ahead, like, kind of both on the data and then also the ML side, so you already mentioned you folks are kind of pulling the products closer together. It's like if you think about 2026, like, what are the challenges you're most excited about, the things you really wanna drive? yeah. It would be interesting to hear that.
[00:13:21] Maddie: Yeah. So I will say that one of the biggest things that we are tackling right now especially more than two years into adopting gen AI and seeing some great success in contextualization, we want to take this to the next level. So Intuit as a whole believes and I truly believe this is tremendously valuable in creating done for you experiences for our users. And what that really means is starting to like create and and um alleviate tasks on our users behalf. Last week we announced something called the debt agent which is helping you, consolidate that or or find options to manage that debt and taking some actions on your behalf. Now none of this honestly is easily done without having and building an agentic data layer. So that's one of the biggest, I think, revelations that not only we are seeing as a company, but also many other other companies are seeing where if you don't structure your data in a semantically, well structured way, you are not likely able to provide the most highly relevant and personalized experiences for users. So, especially because we are part of Intuit and we are part of a broader consumer ecosystem group. One thing that we've been building, in the last year or so it's called the unified consumer profile. And we started this as part of a hackathon. We love doing hackathons here at least twice a year where we really wanted to get to the bottom of how can we create a semantic graph that depicts a financial journey for a user in almost think of it as a tree format, such that entities like debt can be well explained by attributes like balance or interest, and being able to essentially contextualize that and serve it in a highly accurate, real time and consistent way across Credit Karma and TurboTax. And that is ultimately what I believe is one of the biggest unlocks to being able to serve a very personalized experience, at least from a user facing perspective when it comes to GenAI, but also for a wide range of other products. AI in general, but also other product capabilities. So I believe, like, if we don't invest in agentic data layers, may that be user facing or maybe context facing, in session facing, then we will not really be able to truly unlock what we can do, especially with generative AI that contains either small or large language large language models that really need that contextualization to be able to be relevant.
[00:15:52] Benjamin: Right. Okay. And then kind of that basically agentic layer, what is the tech stack for that kind of a may I ask? Because that's obviously an across your product line. Yeah.
[00:16:03] Eldad:
[00:16:04] Benjamin: so it would be interesting. Yeah.
[00:16:06] Maddie: Absolutely. So that is, ultimately based in AWS, and it is something that we are building once more in as possible a cloud agnostic way, but it is part of the broader Intuit stack, which as mentioned earlier is, AWS based. So we are utilizing a lot of the capabilities that Intune has put in place to be able to like ingest the data into AWS to be able to serve the data in real time as both Credit Karma, which is again on Google, and TurboTax, which is on AWS, ingest or serve this information in real time.
[00:16:44] Benjamin: Awesome. Cool. Eldon, any questions from your side? Anything you're curious about?
[00:16:50] Eldad: I think it's crazy how, you know, Intuit I know Intuit. Most of us know Intuit back in the days. Right? Like, so there are people who used to manage their taxes on the Intuit platform, switching products, kind of evolving with Intuit. And now kind of AI is kind of gets context out of all of those system into one place and then getting crazy amazing experience no matter which app you're using. Like, to me, Intuit was always about the software, but, apparently, it's not. So it's all now about AI and the data. And and and Intuit knows us. Right? Like, if you've been running your taxes on Intuit for so long, then you probably get a much better experience continuing to use it because they have the right data. So it's super interesting to see kind of how those platforms evolve and and kind of how crazy value you're getting out of them.
[00:17:47] Maddie: I'm glad that you're you're saying that, and and thank you for being an Intuit user. Indeed, it's been a tremendous journey, and I've seen this for many years. I was back at Intuit more than seventy years ago, and, now here again, one of the core components of what you're describing is something called the generative AI operating system, which Intuit has been investing in tremendously over the last, few years. That also helps fuel and democratize how we use Gen AI across Intuit. So Credit Karma or TurboTax or any other subsidiary of Intuit can tap into this centralized platform and really democratize and adopt Gen AI at scale within its product. And those are the types of examples really that enable us to move fast and continuously disrupt ourselves, especially in the age of AI.
[00:18:41] Eldad: So, Manny, Benjamin, as we all know now, he just recently moved to the Bay Area. And coming out of Munich, he is not acquainted with the whole world of taxes, tech management, and tax education. So I hope that with AI, Benjamin will get a very different experiences and kind of new onboarded user into the Intuit, into the Intuit, platform, Benjamin.
[00:19:08] Maddie: That's the goal. I hope I hope, Benjamin, you you use Credit Karma to build your credit score and credit history. And and use that as an entry point into TurboTax where we can serve you no matter what the complexity of your tax background is. So, hopefully, you take advantage of that.
[00:19:28] Benjamin: Sounds good. I'll let I'll let you know how it goes. And, yeah, kind of excited to learn about this this new world. It's like moving to The US kind of, like, coming from, like, a just different country. Right? Like, it's a lot to take in in the beginning and kind of a lot of things you need to learn about, kind of a lot of new new systems you need to understand. so, yeah, it's gonna be a fun time basically onboarding into all of that.
[00:19:52] Eldad: I think Now when you look to into it, you're having a conversation. Think about it. Like, you previously, you had to learn how to use the UI. Like, ten years ago, like, that was the challenge on on Intuit was protecting you from writing the wrong stuff into your tax process journey. Now it's planning stuff. It's thinking for you. It's it's finessing all of those rough edges and not like, that no one likes to do. So I think, like, this is one of those, like, the auto zoom is completely broken, by the way. It's a new webcam. As you notice, it's still in experimentation mode. let's zoom out. This is the wrong AI. By the way, this is AI driven, the Zoom. and now I've switched to manual mode. I apologize. so many, it's it's fascinating to hear about it, and, we look forward to see where and how it evolves. And it's crazy to see so many engineers are working fearlessly behind the scenes to get that painful process so smooth. What can I say? Well done.
[00:20:58] Maddie: Thank you so much. I'm glad, and thank you for the kind words. And Benjamin, let us know how we can help you in your financial journey and tax journey. And it's a pleasure, speaking with you today.
[00:21:09] Benjamin: Awesome. Thank you. It was great having you on the show. Kind of thank you so much. all the best and looking forward to staying in touch. Thank you. Awesome.
[00:21:18] Eldad: Take care. Bye.
%20logo.png){kind=icon}
