Julie Lerman discusses Object Relational Mappers and Entity Framework with Jeremy Jung. They talk about why we use relational databases with object oriented languages; the benefits of ORM query languages; how specialized databases are reducing the need for ORMs; alternatives to ORMs; using views, stored procedures, and raw SQL with ORMs; the importance of profiling and working with DBAs; how Entity Framework has evolved.
Show Notes
Related Links
- The Data Farm
- Entity Framework
- Pluralsight Courses
- Episode 121 – OR Mappers
- Episode 218 – Udi Dahan on CQRS
Transcript
Transcript brought to you by IEEE Software
Jeremy Jung 00:00:55 This is Jeremy Jung for software engineering radio. Today I’m talking to Julie Lerman, she’s a frequent conference speaker, an independent consultant, and a Pluralsight course author. Today. We’re going to talk about her experience working on ORMs and entity framework. Julie, welcome to software engineering radio.
Julie Lerman 00:01:14 Hey Jeremy, thanks so much for having me
Jeremy Jung 00:01:16 For those who aren’t familiar with forums. What, what are they and why do we need them?
Julie Lerman 00:01:22 The first thing is the definition. So ORM stands for object relational mapper. So it’s a software idea and implementation and in many aspects for taking the concepts that we describe as classes and transforming them into the structure of a relational database. So you can get the data in and out easily. So the ORM takes care of that mapping and the transformation. So you don’t have to worry about writing sequel. You don’t have to worry about getting results, you know, structured results from a database that are in rows and columns, and then transforming them back into object instances. So the ORMs do that for you.
Jeremy Jung 00:02:12 Some of the most popular languages used are object oriented. Why do we continue to store our data in a relational database? One it’s so different from our application code.
Julie Lerman 00:02:24 There’s certainly a lot of reasons for relational databases compared to like document databases, object, databases, relational databases for their structure and normalization and tuning. And it’s different kinds of storage needs. I think then when you’re storing documents, it’s sometimes it’s, I think more about cutting the data back out, finding data. So it’s one thing to be able to store it in a way that makes, but then when you need to pull data together and create relationships that aren’t naturally in those documents, you know, unless you have prepared and designed something like a document database for every possible scenario, there’s also the whole concept of event sourcing where, you know, if you’re just storing the events, then you can actually re persist that data in different structures. So that it’s easier for the various ways you might access it. Also, you know, relational database has been around for a long time. And it’s interesting that even with the popularity and extreme power of these other kinds of non-relational databases to know SQL databases, relational databases are still, I can’t say the source off the top of my head that I’ve, I’ve used them in, in conference slides. They update their data monthly, but I can see these, these graphs where it shows that we’re still about 70% of systems out there are using relational databases and in a lot of cases, so much legacy data too, right? That that’s all going to be in relational databases,
Jeremy Jung 00:04:15 Relational databases. They give us the ability to request just the data we need that, that matches a certain condition, or we can combine different parts of our data and get just a specific thing that we need that if we were just dealing with a bunch of arrays of objects or documents, it might be harder to query for a specific thing. Does, does that sort of match?
Julie Lerman 00:04:43 Yeah. Especially that unplanned data access and another interesting thing. And it’s one of those things that I think you have to like stop and think about it, to realize the truth in it, which is that most systems, most of the work you’re doing with the database is reading the database much more than creating and storing data. That’s the bigger problem, getting the data back out and, and having the flexibility to do weird things. Whereas something like a document database, you know, depending on how you split it up and keys and everything, it’s just more designed. A relational database is just, it’s a little easier to get at that weird stuff. Yeah.
Jeremy Jung 00:05:33 You mentioned ad hoc queries. So with a relational database, you can store all your information and then you can decide later what sort of queries or what sort of information you want to get out. Whereas you had hinted at, with a document database, you have to think upfront about what queries you want to to make. Could you elaborate a little bit? Yeah.
Julie Lerman 00:05:58 I don’t want to be generalizing too much because there’s all kinds of other factors that might come in, especially when talking about huge amounts of data for like machine learning and things like that. There’s a point, you know, the th there’s pros and cons to both of them. And like I mentioned, with event sourcing ways to kind of have your cake and eat it too. When I look for example, at documentation for Azure cosmos DB, that’s what the document database that I have the most familiarity with. One of the most important documents is that upfront design and that design bleeds into how you’re modeling your data in your application so that you can store it fluidly into the document database. So it’s really, really important. Now, I don’t want to conflate this with, you know, the kind of the beauty of something, a system like document database, how that aligns so much more nicely for persistence and most of your, you know, getting data in and getting data out when we’re talking about a domain German design where you don’t want to worry about your persistence, like just focus on the domain and then, you know, we can always figure out how to get the data in and out, right?
Julie Lerman 00:07:22 We’ve got patterns for that, but it is interesting if something like a document database more naturally aligns with your objects, right? So that flows nicely in however, is still really important to be considerate of how you’re breaking up that data in the way you store it, what, you know, what your graphs are going to be, how are you going to, you know, maybe it’s easy to get it in this way, but then what about when you’re getting it out? What about different, different perspectives that you have? So I want to be really careful about comparing and contrasting the document DB versus relational DB. I want to stop at some level of that because there’s always going to be debate and argument about that. However, you know, one of the nice things about an, an ORM is it does take away some of that pain that we normally would have when we’re writing applications and writing that data access layer, where we have to say, okay, I want this.
Julie Lerman 00:08:29 Now I have to figure out the sequel statement, you know, or call a stored procedure, or however I’m going to do it. You know, whether I’m getting data in or getting data out and then getting, you know, and then I have to transform my objects into the sequel, right. Or get data back. And then I have to read through that data and instantiate objects and stuff, all the values into those objects. That’s the stuff that the ORM can repetitive, like bull work, right.writing, writing, all that stuff. So that’s what the ORMs do. Th I think one of the, the next big question that comes up then is yeah, but my SQL is always going to be better than some generic thing that an ORM is going to build for me. Right. The general advice for that is for many cases, what the ORM does will the performance will be just perfectly fine.
Julie Lerman 00:09:32 Right. And then there’s going to be the cases where the performance isn’t good. So then you can, you know, use your own store procedures, use your views. But a lot of the ORM, certainly the one I focus on, which is Microsoft entity framework has the ability to also interact with stored procedures and views and do mappings to views and, and things like that. So, so people don’t realize that, right. That, but maybe 80% of the sequel, that entity framework is going to write whether it’s for queries or for pushing data into the database, maybe 80% of that is perfectly fine. Right. And then 20% of that do performance testing go, yeah, that’s not, I can do better. I can do better. I can do better. I can do better. And fix those up.
Jeremy Jung 00:10:22 I wanna step back a little and walk through some of the features of ORMs that you had mentioned. One of them was not necessarily having to write SQL. So you have some alternative query language that the RM is providing. So what are some of the benefits of that? Like why would somebody not just jump straight to SQL and would instead use this SQL like query language?
Julie Lerman 00:10:51 Well, again, I’d have to kind of focus on.net. And luckily I know you do a little bit of.net programming, so you probably are familiar with link the query language that’s built into.net into, you know, C-sharp and the.net languages. It’s a much simpler expression than SQL, and it’s much more code like, and like, if you were writing a SQL string to put in to your application, we want to use stored procedures in fuse, but let’s say you’re just writing raw SQL, right. You’re writing a string and crossing your fingers that you got it. Right, right. That there’s no typos, but also that you’ve written it correctly. But if you’re seeing something like link, which is part of the language, then you’ve got all the advantages of the language syntax and, you know, IntelliSense. And here’s my object. What are my methods that are available all popping up?
Julie Lerman 00:11:49 So, you know, you don’t have to worry about magic strings, I think really key. And you don’t have to know CQL concepts really. So with entity framework at the very, very beginning, the team actually came out of Microsoft research. The team that had created that started with a SQL like language to do it, but then link was also created at Microsoft by the C-sharp team. And then link was originally created for objects.so you can use that for writing queries, just across your objects. I wouldn’t say link was evolved, but a link version that applied to entity framework was created. So then, and there was also linked to sequel which was a much lighter weight or that was also created within Microsoft. So you’re using the same, almost the same link. A lot of the methods are shared. Some are more specific to entity framework.
Julie Lerman 00:12:49 There’s another benefit. Like I use link for all kinds of things in my code, not just for database, the database interaction with entity framework. I use link just when I’m working with a raise, right. Working with data in objects, in memory. So I’ve also got this common syntax. I already know how to use it, right? So this is just some special features of it that are specific to using entity framework. So there’s all kinds of benefits. And the best benefit was it enabled me to essentially forget SQL. Like I forget how to write T SQL. If I have to write raw SQL, I have to Google it right after all these years of coding. But I don’t mind
Jeremy Jung 00:13:30 With link. This was a query language that was already built into C-sharp and it also envy vb.net. Right.so, so all of the languages that run on the dotnet runtime, and so the developer may already be familiar with this query language and they can apply it to making queries to a database rather than having to learn sequel, which would be an entirely new language to the developer.
Julie Lerman 00:14:09 Yes. And not just that, but I don’t have to learn T sequel and Oh, sequel and PC equal. Right. So the different providers, so there’s like all kinds of different database providers for entity framework. And we have, you know, the same for other ORMs. I feel a little bad just focusing on EOF, but that’s by area of expertise, but the providers are the ones that do that transformation. So entity framework, iterates over the query a few times and creates a query tree. And then the query tree is passed onto the provider, which then does the rest of the transformation into its particular sequel. Right? So I actually go back and forth between SQL server, SQL Lite and Postgres, those SQL servers, what I’m most familiar with, but I can use SQL Lite and Postgres. And there’s a lot of generic, common stuff. Right? So as a matter of fact, the way link for entity framework is built.
Julie Lerman 00:15:18 That’s just one thing. And then the providers figure it out. So if you need something that’s really specific to SQL server that doesn’t work in Postgres or something in Postgres that doesn’t work in SQL Lite, there might be an extension that lets you tap into that particular thing, right. Or, you know, or call a stored procedure, except you wouldn’t do that in SQL Lite because they don’t have stored procedures. Interestingly, kind of tapping back to talking about document databases. There is a provider for Azure cosmos DB. Okay. So we’ve got an object, relational mapper that maps to a non-relational database. So the reason that exists is because there was a lot of people that said, I really am familiar with entity framework. I’ve been using it for years. We’re going to be using cosmos DB can’t I just use that. So that exists. And Azure has a sequel.
Julie Lerman 00:16:24 I don’t want to say language. Oh, you know, can you, sequel has SQL I’ll just say SQL language for accessing, but you can also there’s other API APIs that are used also. It’s not all SQL, but because it has the CQL construct, you know, the providers kind of already know how to build CQL. So it’s just well build the SQL that Azure works. So it’s not solving the problem that we have with relational databases, which is, you know, that we’ve got this rows and columns structure and we need to do this translation all the time. The problem it was solving that provider’s solving by linking entity framework and the document database is I already really know how to use entity framework. And here I am just, I’m just doing some queries and persisting some data. Why do I have to learn a whole new API?
Jeremy Jung 00:17:16 You’re you’re saying that even with a document store, you might be able to use the same query language that’s built into your, or with a relational database you have joins and things like that that allow you to connect all your tables together and do all these different types of queries. Whereas with a document database, I usually associate those with not having joins. Is that also the case with this document database that you’re referring to with, with cosmos DB,
Julie Lerman 00:17:50 I’ve written only tiny little minimal CQL for accessing, it used to be called document DB, cosmos DB directly. So can’t off the top of my head. Remember how that CQL works that out. If you’ve got, you know, if you’ve got to pull multiple documents together, I don’t know, but what’s nice. Is that in your code? You don’t have to worry about that stuff. Oh God. Inner joins and outer joins and all like, Oh my God, I can’t, I can’t write that stuff anyway. So link makes sense to me. I know it really well. Right. So I don’t have to worry about that.
Jeremy Jung 00:18:30 It sounds like at a high level, you get this query language that’s built in to the language is more similar to the other code you would write. And you also get these compile time guarantees auto-completion so that you know what things you’re allowed to call versus SQL, where, like you were saying, it’s you have this giant string in you. You hope that hope you got it right. And don’t really know until you query. So there’s a lot of benefits with an ORM. You were also talking about how, when you don’t have an ORM, you have to write your own mapping between after you’ve done your SQL query, you get some data that’s in the form of just a set of records and you have to convert those into your objects that are native to your application. And so basically the RM is, is taking care of a lot of the, you could say repetitive or busy work that you would normally have to do
Julie Lerman 00:19:32 To be fair. There are mappers, there are programs that are mappers or APIs that are mappers like auto mapper for.net, right? So the connection string, like I don’t have to worry about the connection, you know, creating connections and all that kind of stuff, either with it, with the ORM. But you know, something like auto mapper, you can have a data model that really matches your database schema, and you can have a domain model that matches the business problem you’re trying to solve. And then you can use something like auto mapper to say, okay, like here’s what customer looks like in my application. Here’s what customer looks like in this other data model. That’s gonna go to my data. But that map matches my database really easily. And auto mapper, lets you say, this goes with this, this goes with that blah, blah, blah, blah, blah.
Julie Lerman 00:20:25 So that stuff does exist. And, and there are also other, you know, lighter much equity framework is a big thing. It does a lot. There are certainly lighter weight or M’s that they don’t do as much of that takeaway as much of that work. However, if you need to squeak every little millisecond out of your performance, there’s an open source or I’m called dapper. And it was built built by the team that writes stack overflow because they definitely needed to squeak every millisecond of performance out of their data access. There was no question about that. So with dapper, it takes care of some of the stuff for you. Some of that creating the connections and materializing objects for you, but you still write your own sequel because they needed to tightly control the CQL to make sure because it’s more direct. It doesn’t have lots of things happening in between that entity framework does. And there’s lots of, you know, dials and entity framework where you can impact how much work it’s doing and how many re how much resources it’s using. So
Jeremy Jung 00:21:49 My understanding with whether you would use entity framework versus one of these smaller or ums or a more basic mapper is it sounded like performance. And so it sounds like maybe you would default to using something like entity framework and then only dropping down to something like dapper, if you were having performance issues, is, is that where your perspective comes from?
Julie Lerman 00:22:21 Wait until I’m having performance issues. I think people sometimes will make that decision in advance, but in an interesting plan of attack, you know, so that you can benefit from the simplicity of using entity framework. So you don’t have to write the SQL, et cetera is for example, using kind of a CQRS pattern, right? So, you know, so you’re separating the commands and the queries separating pushing data into the database and the queries. Cause it’s usually the queries where you need to really, really focus on the performance and worry about the performance. So some people will use CQRS and let entity framework take care of all the writing and let dapper take care of all the readings. So they designing their applications so that things are not, not the business logic, but you know, the, the layer of the application that’s taken care of the persistence. So designing it so that those are completely separate. So I’m doing writing, so we’re spinning up dapper and we’re going to, or actually the other way around, I’m doing I’m writing. So I’m going to spin up entity framework context and, you know, say, go ahead and save changes or I’m doing a query and I’m writing my own sequel because I trust it. Right? So now I’m going to spin up dapper and execute that SQL and let it create the, create my, for me.
Jeremy Jung 00:23:46 So it sounds like that might be, it might be better to actually decide that up front is figuring out what is the amount of load I’m going to get in my application? What are the types of queries I’m going to be using for writing and reading data? And then from there, figuring out, should I use dapper, should I use entity framework or
Julie Lerman 00:24:12 No amount of testing and exploration, you would need to do proof of concept stuff. I think you might want to do first because I think entity framework, query performance, surprises people a lot, like it’s a lot better than people presume it’s going to be. So yeah, I would definitely do some proof of concept before going that route. Cause it’s, it’s certainly a lot easier to just use one thing all the way through. However, you know, if you’ve got things, you know, applying separation of concerns and you know, things are broken up, then it’s not as traumatic to say, you know, for this bit it is running a little slowly and let’s just use dapper, but it’s, if it’s separated out, then it’s easier to just say, yeah, let’s just, you know, let’s just change over to that. Being our infrastructure.
Jeremy Jung 00:25:05 Are there workloads in a general sense that you would say that entity framework performs worse than, than others? For example, an app with a heavy right workload, you know, would that be an issue in entity framework? Just, are there examples of specific types of applications?
Julie Lerman 00:25:27 I don’t know if it’s workload compared more to structure your schema or what it is you’re trying to get at, you know, sometimes it’s how your model is designed, right? So there are just some places that any frameworks not going to, just not going to be able to do something as well as your own SQL. I had an example this was years ago. So entity framework was not in its infancy, but it was like . And I had a client who, well, they had gone down kind of a not great path of implementing entity framework in, in their software. That’s used across a big enterprise. They had created one big, huge model for everybody to use. So that Mo one model was responsible for, you know, no matter what you were doing, you had to use that model. Just pulling that model into memory was a lot of work for entity framework, but it created a lot of problems for the teams who were stuck using that because of the way they had set that up.
Julie Lerman 00:26:32 They had this common query that would only return one piece, one row. They were looking for one row, but because of the way they had designed this model entity framework had to go through all kinds of all kinds of hoops to create the CQL to get at this one piece of data, to, to write this query. And it was taking, it was an Oracle is taking two minutes to execute and they’re like, what are we doing? And I looked at us at just write a stored procedure, do not, you know, like you can’t, it’s too late to change this model because everything’s dependent on it. So just write a stored procedure and start procedure took nine milliseconds. So like, okay, we can, you know, we can pay you for the whole three days. You can go home now, if you want, that was worth it.
Jeremy Jung 00:27:19 Yeah. And so it sounds like though, that’s not so much a choice of not using entity framework as it is. You’re still using entity framework, but you’re using it to make a request to a stored procedure in the database, similarly to how you could probably send Ross equal from entity framework.
Julie Lerman 00:27:41 Yes. So you can still take advantage of that. So we’re still using entity framework and saying, Hey, I don’t want you to write the SQL, just execute this for me. And so that’s still taking advantage of the fact that it knows that SQL ser, you know, knows SQL server. It knows what the connection string is, et cetera. It’s not writing the query now there’s we can take advantage of views in the same way and can actually map our objects to views and have it. So we’re still executing the view and it’s going to still materialize the data for us, which is really nice
Jeremy Jung 00:28:56 Yeah. So when you were talking about deciding whether or not to use entity framework, when you use entity framework, you could still be using a mix of the, the link query language, maybe raw SQL commands. Like you said, views stored procedures. It’s not this, this binary choice where you’re saying, I’m going to write everything myself, or I’m going to leave everything density framework.
Julie Lerman 00:29:24 Absolutely. I mean, that’s yeah. The way we approach programming overall. Right, right. So I’ve got all of these wonderful arrows in my quiver to choose from. Right. And, and I not, I’m not stuck with one. I just leverage, you know, using my knowledge and my experience making decision, Hey, this one, this tool, right. Tool for the job. Right. Like that’s what we, you know, we’ve been saying for decades, not just programming, using the right tool for the job, but just because you want to use any framework doesn’t mean you have to use it all the way through, but I think that’s, you know, that’s like, again, you know, using, using C-sharp for this and, you know, maybe I’ll use no JS or maybe I’ve got some really complex mathematical thing to do for this particular thing. I’ll use F sharp or, you know, some other functional programming links.
Jeremy Jung 00:30:20 So we did a show back in 2007 about RMS, and I know that you’ve had a lot of experience, both working with ORMs and, and probably working with applications without them just writing SQL directly. And so I would imagine there’s been a lot of changes in the last decade, or maybe even couple of decades. What are some of the big things that you think have changed with RMS and the last 10 or 20 years?
Julie Lerman 00:30:51 Well, I think something we’ve been talking about, which is no SQL database has reduced the need for ORMs right. Just slice that right off the top. There’s so many scenarios where people don’t need relational databases to solve that. And also, so this, this interesting, right? Because I, I’m not sure that the answer is about what’s happened with the ORMs. I think the answer is more, what’s evolved with the way we persist data and the way we perceive data machine learning, right? Like the idea of event sourcing, right. Talk about right. Tool for the job. Like here’s my data, but for this, this scenario, it’s going to be more effective and efficient for me. If that data stored in relational for this scenario, it’d be more effective if it’s stored as documents, right. For this scenario, it’d be more effective if it’s texts like just a text file.
Julie Lerman 00:31:59 Oh, I forgot XML. Just kidding, or Jason. Right. What, however, so I, I, I think that’s more, what’s impacted RMS than the ORMs themselves. And I know, you know, entity framework is still going strong and evolving and very, very wide use in the dotnet space. Some of the ones that were more popular are not really around anymore or not, haven’t been updated in awhile. Dapper gets its love of course, from the stack overflow team. So I really think it’s more about the change in how we store data. That’s just reduced, people’s need for RMS, but still like whatever that resource was, you know, so 70% of the applications are still using relational. So, so there we go. And then, and then there’s the people like, no matter what, like we will never use an ORM. We will always use, you know, write our own SQL or, you know, long of this legacy databases. They have beautiful stored procedures and views and you know, finely tuned databases. Right. And they’ll write the SQL for you. These people just don’t ask me to do it cause I will never, I’ll probably never write SQL that’s as good as what entity framework with its algorithms and command trees can figure out on my behalf. But there are plenty of database
professionals who can definitely do that
Jeremy Jung 00:33:42 For somebody who is working with a relational database. And let’s say they worked with entity framework, or they worked with hibernate or any number of forums maybe a decade ago, is there anything that’s changed where you would say, Hey, this is different. Maybe you should give this a second. Look.
Julie Lerman 00:34:05 And again, I apologize for not being able to speak to hibernate, like what’s happened to, you know, how, how that has evolved, but when Anthony framework first came out and this was very different than an hibernate, like the.net version of hibernate entity framework did not have any concept or, or appreciation or respect for our objects. It was really designed from the perspective of storing relational data and it wasn’t testable. It just didn’t have any patterns. Well, interestingly, I spent a lot of time researching how entity framework aligns with good model design based on how we design, like for domain-driven design aggregates. And some of the things that we pull in, put in place to protect our entities from bad things, to people doing bad things with them. And you know, so any framework at the beginning did not give us any way to do that.
Julie Lerman 00:35:08 It really locked you into it. Didn’t just lock your data, access in to entity framework, the whole framework, it locked your entire application in. So like your, your, your classes, all your entities and your classes had to be completely tied to entity framework in order for entity framework. Now that’s all decoupled, right? So entity framework also we had this big model, this big visual modeler that in order to create the model, you had to start with an existing database. Now it’s like, look, here are my classes. I don’t want to design a database. I’m designing software, right? I’m solving business problems. I have my domain model. So you build your domain model and then say, Hey, entity framework. Here’s my domain model. Here’s my database. Oh. And by the way, I know that by default, you’re going to want this to go there and this to go there, but I need to, I need to tell you, there’s a few things I want you to do a little differently.
Julie Lerman 00:36:11 So we have all of that and we tell entity framework has nothing to do with our domain logic. So it’s gotten better and better and more respectful of our code, our own, you know, our domain logic and our domain business logic and code and staying out of the way. And that’s been a big deal for me as you know, somebody focused on domain-driven design, which is when, as I’m building my software, I do not want to think about my data persistence, right? It has nothing to do with my domain.I like to joke unless I’m building the next Dropbox competitor, right? So data persistence has nothing to do with my domain. So I look to see how well as each iteration of entity framework comes out, I look and do some experiments and research to see how well it it’s default understand or map correctly, like get my data in and out of my database without me having to tweak entity frameworks mappings.
Julie Lerman 00:37:16 But more importantly, without me having to do anything to my domain classes in order to satisfy entity framework and with each iteration, it’s gotten better and better and better, which is nice. Now, one, one bit of pushback I had in the early days when I was talking about entity framework and DDD was well, you’re mapping your domain directly to the database using entity framework. You shouldn’t do that. You should have a data model and map that, but what’s interesting is entity framework becomes the data model, right? It has all these mapping, the mappings of the entity framework become the data model. So I’m not doing it directly. That is the data model. So I, it is solving that problem. It’s giving me that separation and there are still some design, some aggregate patterns that entity framework, for example, won’t satisfy in terms of the mappings, or maybe just some problems, some business problems or storage problems that you have there, there are some times when it just makes sense to, you know, have something in between even if the entity framework is still in between that, like I have a specific data model because it just handles it differently.
Julie Lerman 00:38:34 So you don’t have to change how your, your domain, I’m very passionate about that. Don’t change the domain to make entity framework happy. Never, never, never, never.
Jeremy Jung 00:38:45 Could you give a specific example of before the objects you would have to create in order to, to satisfy entity framework? Like what, what were you putting into your objects that you didn’t want to put in
Julie Lerman 00:39:00 That it didn’t want to? I don’t want to, well, one of the, one of the things that any frameworks mappings could not handle is if you wanted to completely isolate or encapsulate a collection in your entity, in your, in your class, if you want to encapsule if you want to completely encapsulate the collection entity framework, couldn’t see it, right. Entity framework only understood for example, collections, but maybe you want to make it so that you can only iterate through it, enumerate through it. Right. But if it was an I innumerable, which is so that’s what you would do like in your class, you would say, Oh, I’m going to make it a nice innumerable. So nobody can just add things to it and go around my business logic, which is, if you want to add something, I have to go over to this business logic to make sure all your invariants are satisfied or whatever else needs to happen.
Julie Lerman 00:39:58 So entity framework literally could not see that. And so we struggled with finding ways to work around that adding stuff, or just giving it up and saying, okay, I’m exposing this here, please. Don’t ever use it that way, please, please. But you know, you have to tell the whole team don’t ever use it that way, right? Like, ah, but with any framework, core three, they changed that. So I could, you could, you can write your class the way you want to, and you can use the patterns that we use to encapsulate collections and protect them from abuse, misuse, and still entity framework is able to discover them and understand how to do all the work that it needs to do. So that’s just an example,
Jeremy Jung 00:40:46 What you want it to do in your domain object is make this collection be an innumerable, which makes it so that the user of that class is not allowed to add items to it, right.
Julie Lerman 00:40:59 The developer, right. Developers using it add or remove
Jeremy Jung 00:41:04 Right. Add or remove. Andso you want it to be able to control the API for the developers, say that you want to receive this object and you can, you can see what’s in it, but you can’t add or remove from it.
Julie Lerman 00:41:20 Well, you can, but not directly through that property. Right. Right. You’d have to use a method that has the controls. I refer to it as add D D add design, right. Because I am a control freak. I want to control how anybody’s using my age.
Jeremy Jung 00:41:39 Yeah. And you wanted to control how people use the API, but if you did that within your object, then entity framework would look at that high innumerable and say, I don’t know what this is. So I can’t map this to the database,
Julie Lerman 00:41:53 Just completely ignore its existence. So that was one evolution of entity framework that you can do that now.and all kinds of all kinds of other things.
Jeremy Jung 00:42:05 Are there any other specific examples you can think of, of things that used to be either difficult to do or might frustrate people with entity framework that, that are addressed now?
Julie Lerman 00:42:16 Sure. Oh, well, along those same lines of encapsulating properties, right? So, and capsulated collections was really, really hard. Like it wasn’t hard, it was impossible. But if you want to either encapsulate properties or not even expose properties, you didn’t want to expose them all at all.and you just wanted to have private fields. You couldn’t do anything like that either. But now with this more modern EOF core, we could, you could literally have an object, a type defined with fields, no properties exposing those fields outside of the API. But, and, but you can still have, you can still persist data. Entity framework is still aware of them or depending on how locked down. Like if you made them totally private, you have to tweak the configuration entity framework, you telling a farmer, Hey, this field here, I want you to pay attention to it.
Julie Lerman 00:43:17 Right? So there’s some conventions where it will automatically recognize it. There are some ways of designing in your type that any framework won’t see it, but there’s, you can override that and tell entity framework about that. So I think that’s a, that’s a really interesting thing because there’s a lot of scenarios where people, you know, want to design their objects that way. And you know, again, just so much of that is about protecting, you’re protecting your API from misuse or abuse, but, but not, not putting the onus on the users of your API, right. The API will let you do what you’re allowed to do, and it won’t let you do what you’re not allowed to do.
Jeremy Jung 00:44:01 You had mentioned earlier, another big difference is that it used to be that you would create your database by writing SQL. And then you would point entity framework to that to generate your models.
Julie Lerman 00:44:15 Oh yes. The very, the very first iteration, that was the only way I knew how to build a model. Oh, wait, there was a way to build the model, a data model in the designer. That’s all right. But then, you know, everything’s dependent that model or on the database instead of, no, I want to write my domain logic. Now, any framework, not everybody has a brand new project, right? But say you get a brand brand new project and there is no existing database. So you can ask entity framework to generate the database based on, you know, the combination of its assumptions from reading your types and extra configurations, you add to entity framework and create the database. Right? So there, I hear all the DBS go like, Oh no, no, no, no, no. That’s not how it works. Right. But this is, this is the simple, the simple path.
Julie Lerman 00:45:10 Like, so first you can have it create the database. And then as you make changes to your domain and any framework has this whole migrations API that can figure out what has changed about your domain and then figure out what SQL needs to be applied to the database in order to make those changes. So that’s, that’s one path a twist on that path of course, is you can create that stuff. Instead of having those migrations, create the database or update the database, you can just have it create sequel and hand it to the professionals and say, this is kind of, sort of what I need. Right? You can do this and then just make it the way you want it to be just as long as the mappings still work out. But the other way is, you know, if you do have a Greenfield application and you do have a database already, you can reverse engineer just like we did at the very beginning, but it’s a little different, but you can reverse engineer into a data model, which is your classes and a deep context, the entity framework configuration.
Julie Lerman 00:46:20 So the classes are, I refer to those as a stake in the ground, right? Because then you can just take those classes and make them what you want them to be. And then just make sure that you’re still configuring the mapping. So everything lines up. So there are two ways to go still. But at the beginning it was just really, I mean, there was this, I forgot even that the designer had the entity data model designer had a way to design a model visually and then create both the database and the classes from it. But that, that went away really quickly. So it was essentially, you know, point to the database, the whole huge database and create one whole huge data model from it. And that’s what, everything, how everything would work now, a we don’t do that. And also we still don’t have one big, huge data model. Right. We separation concerns again. So we have a data model for this area of work and a data model for that area of work. And, you know, they might even point back to their own own databases. Right. And we started talking about microservices and et cetera.
Jeremy Jung 00:47:24 Yeah. So if I understand correctly, it used to be where you would either use this gooey tool to build out your, your entities or your database, or you would point entity framework at an existing database that already had tables and columns, all configured. Right.
Julie Lerman 00:47:44 And all of that UI stuff has gone. There’s. I mean, some third-party providers make some UI stuff, but as far as the enemy framework has come, you know, Microsoft is concerned, we’ve got our classes and we’ve got our database. There’s no UI in between.
Jeremy Jung 00:48:02 It sounds like the issue with that approach was that it would generate your domain models, generate your classes, but it would do it in a way that probably had a whole bunch of entity framework, specific information in those classes,
Julie Lerman 00:48:19 In your classes. Yeah. So much dependencies in your classes. Now, none of that, that actually, they started moving away from that in the second version of the entity framework, which was called the not two, but that’s because it was aligning with.net for, you know, but now we’ve now we’ve got EDF core, which came with, along with.net core, which is, you know, cross-platform and lightweight and open source.
Jeremy Jung 00:48:48 So it sounds like we’re moving in the direction from having these domain models. These classes have all this entity framework specific stuff into it to creating just basic or normal code a normal COVID. Yeah.like the kind of code you would write, if you weren’t thinking about entity framework or really thinking about persistence specific issues now with current or like entity framework core, those are able to map to your database without, I guess the term could be polluting all your classes.
Julie Lerman 00:49:27 Yes. And to be fair entity framework was that was the evil doer at the beginning of entity framework. Right. And hybrid hibernate didn’t do that. And hibernate didn’t do that. They totally respected your domain classes and it was completely separate. So that was the big, there was a big, I’ll just say a big stink made by the people who had been using these kinds of practices already and were using tools like an hibernate. And a framework came around and said, Oh, here’s the new RM on the block from Microsoft. Everybody needs to use it. And they looked at, and they’re like, but, but, but, you know, because entity framework was inserting itself into its logic into the domain classes. Right. And the anti framework team got a lot of big lessons from the community. That was really a where of that. And I wasn’t right.
Julie Lerman 00:50:25 I was, it was all new. I had used an ORM before, so it was like, Lottie, Dodd, this is cool. This is nice. And you know, all these other people were like, like, this is terrible. Why is Microsoft doing this? And like, I don’t understand. Right. So eventually it evolved. And I learned so much, I learned so much because of them. I mean, these are really smart people. I have a lot of respect for. So I was like, okay, they’re obviously know what they’re talking about. So I gotta, I gotta figure that out. But that’s how I ended up on the path of learning about DDD. Thanks to them. Yep.
Jeremy Jung 00:50:58 It sounds like there’s been three versions of entity framework. There’s the initial version that tried to mix in all these things into your domain models. Then there is the second version which you said was called entity framework for that, that fixed a lot of those issues. And then now we have entity framework core. I wonder if you could explain a little bit about why a third version was created and what are the key differences with, with core?
Julie Lerman 00:51:29 Yeah. So this is, this was when Microsoft also at the same time, it taken.net framework and rewritten it to be cross-platform and open source and use modern software practices. And at the same time, the same thing happened was done with entity framework. They, they kept many of the same concepts, but they rewrote it, you know, because the code base was like 10 years old. Right. And also just in old thinking. So they rewrote it from scratch, keeping a lot of the same concepts. So people who were familiar, you know, who’ve been using any framework, they wouldn’t be totally thrown for a loop. And, you know, so rewrote it with modern software practices, much more open API in terms of flexibility and open source. And because it was also part of dotnet core cross-platform. So that was huge. And then they’ve been building on top of that.
Julie Lerman 00:52:33 So entity framework, the original entity framework. So I think it’s interesting how you say there were three. So the first iteration of entity framework, and then this next one where they really honored separation of concerns. So that was then they kind of went to a new box with F Corps and served with the F core one and then two and three and no four. And now it’s five. So skipping a number again. So ESX is still around because there’s millions of applications that are using it and they’re not totally ignoring it. As a matter of fact, they brought on top of dotnet core. So ESX is now cross-platform, but it’s still in so many applications and they they’ve been doing a little tweaks to it. But if core is where all the, all the work is going now. Yeah.
Jeremy Jung 00:53:27 And it sounds like core is not so much a dramatically different API for developers, but is rather an opportunity for them to clean up a lot of technical debt and provide that cross-platform capability at least initially. Yeah.
Julie Lerman 00:53:45 And enable it to go forward and to do more, be able to achieve more functionality that people have wanted, but was just literally wasn’t possible with the earlier stack.
Jeremy Jung 00:54:00 Do you have any examples of a big thing that people had wanted for a while they were finally able to do?
Julie Lerman 00:54:06 Yes. And this is really, really specific now because you know, this is about entity framework with the newest version F core five, that’s coming out. One thing that people have been asking for since the beginning of the F time was when you’re eager loading data. So there’s a, a way of saying, you know, bringing related data back in one query does actually two ways, but there’s one way called a method called include. So including is so that you can bring back graphs of data instead of bringing, you know, go get the customers, now, go get their orders. And now we need to merge them together in memory, in our objects. So the include method says, please get me the customers and their orders, and include transforms that into like a join query or something like that brings back all of that different data and then builds, builds the objects and make sure they’re connected to each other and memory that’s a really important thing to be able to do. The problem is because it was all or nothing, whatever relation, whatever related data you were bringing back in the include method, you would just get every single one. So, you know, if you said customer include their orders, there was no way to avoid getting every single one of their orders for the last 267 years
Jeremy Jung 00:55:29 Able to do and include. So you’re able to join a customer to their hoarders. But in addition to that also have some kind of filter, like aware clause
Julie Lerman 00:55:40 On that, on that, that child data and say, you know, I’m doing air quotes, that child data, that related data. So we could never do that before. So we had to come up with other, you know, other ways to do that.
Jeremy Jung 00:55:53 So it was, it was possible before, but the way you did it was maybe complicated or convoluted
Julie Lerman 00:55:59 And it, and it writes good SQL for it too. Yeah. You know, better than me, maybe not better than, you know, many of my really good DBA friends, but definitely better than, yeah.
Jeremy Jung 00:56:11 Probably a bit better than the majority of developers who know a little bit of SQL, but are definitely not DPA.
Julie Lerman 00:56:20 Yeah. I mean, that’s one of the beauties of a good ORM,
Jeremy Jung 00:56:26 I guess it, it doesn’t quite get you to being a DBA, but it levels the playing field somewhat
Julie Lerman 00:56:33 Well.if I, I can hear my DBA friends cringing at that statement, so I will not agree with that. It doesn’t level the playing field, but what it does is it enables many developers to write pretty decent SQL how’s that? How’s that my DBA friends.
Jeremy Jung 00:56:56 Yeah. I mean, I guess that’s a interesting question in and of itself is that we’re using these tools like ORMs, that are generating all this code for us. And we, as a developer may not understand the intricacies of what it’s doing and the SQL it’s writing. So I wonder from your perspective, how much should developers know about the queries that are being generated and about SQL in general when you’re working with an aura?
Julie Lerman 00:57:28 Well, I, I think it’s important, you know, if you don’t have access to a professional data professional it’s important to at least understand how to profile and how to, you know, recognize where performance could be better and then go seek a professional. Right. But yeah, profiling is really important. So if you’ve got a team where you do have a DBA summit, somebody who’s really good with SQL and understands, they should be doing the pro, they can be doing the profiling, right. Capturing it. And there’s, you know, whether, if they’re using SQL server, they might just want to use SQL server profile or there’s all kinds of third-party tools.there’s some capability built into entity, into visual studio if you’re using that or there’s, there’s all kinds of ways to profile, whether you’re profiling the performance across the application, or you want to hone in on just the database activity. Right? Cause sometimes the, if say there’s a performance problem, is it happening in the database? Is it happening in memory? Is entity frameworks trying to figure out the SQL, is it happening in memory after the data has, you know, the database did it really fast? The data’s back in memory now entity framework is chugging along for some reason, having a hard time, materializing all this objects and relationships. So it’s not just about not always about profiling the database. And there are tools that help with that. Also
Jeremy Jung 00:59:04 You have the developers writing their queries and something like entity framework. And you have somebody who is more of an expert, like a DBA who can take a look at how are these things performing. And if there are issues they can dive down into those queries, tell the developer, Hey, I think you should use a stored procedure here or a view that, that sort of thing.
Julie Lerman 00:59:30 Yeah. Or, you know, even kind of expand on that. Like just if you know, unless you’re solo right. Or a tiny little shop, these are how you build teams. You’ve got QA people, you’ve got testers, you’ve got data experts. You’ve got people who are good at solving certain problems and everybody works together as a team. For sure. Yeah. I’m such a Libra. Why can’t we all get along?
Jeremy Jung 00:59:56 Yeah. And it sounds like, I mean, it takes care of a lot of the maybe repetitive or tedious parts of the job, but it does not remove the need for, like you said, all these different parts of your team, whether that’s the DBA who can look at the queries or maybe someone who is more experienced with profiling. So they could see if object allocation from the CRM is a problem or all sorts of different, more specific issues that they could dive into. A lot of these things that, that an RM does. You were saying, it’s about mapping between typically relational databases to objects and applications and something you hear people talk about. Sometimes there’s something called the object, relational impedance mismatch. And I wonder if you could explain a little bit about what people mean by that and whether you think that’s a, that’s an issue.
Julie Lerman 01:00:57 Well, that is exactly what our abs are trying to solve, aiming to solve. The mismatch is the mismatch between the rows and columns and your database schema and your, the shape of your objects. So that’s what the mismatch is. And it’s an impedance for getting your data in and out of your application. So the ORMs are helping you solve that problem instead of you having to solve it yourself by, you know, reading through your objects and transforming that into SQL, et cetera, et cetera, all those things we talked about earlier. I haven’t heard that term in such a long time. That’s so funny to me. Like I, it’s funny that, you know, we talked about it a lot when, and like in the world, that was like what CF for it’s to take care of the impedance mismatch. Duh. No, we never said dumb. Just kidding.yeah, so, but we, we talked about that a lot at the beginning of entity frameworks history, and yeah, I haven’t had anybody bring that up in a long time, so I’m actually grateful. You did.
Jeremy Jung 01:02:00 So maybe we’ve gotten to the point where the tools have gotten good enough where people aren’t thinking so much about that problem because ideally entity framework has solved it,
Julie Lerman 01:02:13 Any framework and hibernate dapper. I think that’s a, a good place to,
Jeremy Jung 01:02:20 To wrap things up. So if people want to learn more about what you’re working on or about entity framework or check out your courses, where should they head?
Julie Lerman 01:02:32 Well, I spend a lot of time on Twitter and on Twitter, I’m Julie Lerman and I have a website and a blog. Although I have to say, I tweet a lot more than I blog these days, but my website is the data farm.com and I, I have a blog there and of course look me up on plural site.
Jeremy Jung 01:02:54 Cool. Well, Julie, thank you so much for chatting with us.
Julie Lerman 01:02:57 It was great, Jeremy, and you asked so many interesting questions and also helped me take some nice trips into my entity framework past to remember some things that I had actually forgotten about. So that was interesting. Thanks. Hopefully. Good memories. Yeah. All good. All good. All right. Thanks a lot, Julia. Thank you.
Jeremy Jung 01:03:19 This has been Jeremy Jones for software engineering radio. Thanks for listening.
[End of Audio]
This transcript was automatically generated. To suggest improvements in the text, please contact [email protected].
SE Radio theme: “Broken Reality” by Kevin MacLeod (incompetech.com — Licensed under Creative Commons: By Attribution 3.0)
