Laravel Performance
This episode is for members only

Sign up to access "Laravel Performance" right now.

Get started
Already a member? Sign in to continue
Playing
13. Chunking with Eloquent

Episodes

0%
Your progress
  • Total: 1h 9m
  • Played: 0m
  • Remaining: 1h 9m
Join or sign in to track your progress

Transcript

00:00
If you need to iterate over a huge amount of records for any reason, you're going to want to make use of chunking. And we're going to take a look at this
00:07
with a huge amount of posts in the database. So I've gone ahead and updated the blog seeder with 20 users that each have 10,000 posts. Probably unlikely, but we want a huge
00:19
amount of data here that we can test this out with. OK, so let's imagine that we're building out a console command in here. So let's use Artisan to make a command and say we want to create out a export posts command that we can run through Artisan.
00:36
So let's head over to that export posts command under app console and commands. And let's go ahead and just grab the name of this. In fact, we can change this around. So let's say posts and export.
00:48
Let's grab the name of that so we can use it in the command line. And at the moment, of course, this just does nothing. Let's just run it anyway. So we've got it in our console memory.
00:57
And in here, what we want to do is grab every single post from the database that's 200,000. And we want to iterate over them and do something. So how might we do this usually?
01:08
Well, we would probably go ahead and start either assigning and we would go ahead and say post get and we could either do a for each loop over these. So just put in the namespace for post real quick. We might say for each post as post and then we might do something in here.
01:24
So just as an example, let's go ahead and say this info and we'll just grab out the post ID just so we can see it in the console to test this out. Now I'm going to go ahead and run this. And what's actually happening is now Eloquent is fetching all of them records
01:39
because we're just using get here and it's putting them into memory. So you can see, first of all, this is taking a while to run. And what's actually happened here, although we don't see an error in the console itself, is this is run out of memory.
01:53
We don't see any output here at all. Nothing has happened and it's taken 10 seconds to run. So obviously this isn't working. Another alternative to a for each loop is to go ahead and use each because we get
02:04
a Laravel collection back when we use post. So we can go ahead and do this and we could say this info and post ID. And for each of these iterations, we'll get in here a post like so. OK, so let's run this.
02:20
We get exactly the same thing because effectively behind the scenes, this is just doing a for each loop over each of these records and nothing's happening again. This is going to take about 10 seconds. It's going to run out of memory and we don't see any output whatsoever.
02:34
So really, we cannot do this with this amount of records and even with a much smaller amount of records. It's probably not a good idea either because you're going to run into huge memory issues, particularly if you're running a command like this very frequently.
02:47
So how do we resolve this? What we can actually do is make use of chunking. So we're going to go ahead and get rid of all of this. We can actually get rid of the assignment
02:55
here because we don't really need this anymore. And instead of just getting all of these records, which is going to give us back 200,000, we're going to go ahead and use this chunk method.
03:05
What this will allow us to do is provide a value in here. And Laravel will fetch a thousand records, do something with or give you the opportunity to do something with these records, then it will clear them out of memory and it will give us the next 1000.
03:21
Again, we can export each of them posts. It will clear that out of memory. And again, it will grab the next 1000 until it has gone through every single post in the database or whatever data
03:32
you're working with. So let's look at how we actually use this. We define a closure here as the second argument. And in here we don't get a post itself. We get a collection of posts because we're grabbing the first 1000,
03:46
not iterating over each post. We're chunking this and getting a collection for each of these. So if we just call this post and just die dump on posts, what we actually see here when we run this is we get a collection.
03:58
We had up or reduce this amount quickly. We get a collection of posts, so let's just use get class in here just to actually show you that we get back a collection. OK, so now what we can do is because we're only grabbing a small amount here,
04:19
we can then do something like posts each because each is a method on a collection. Or if you really wanted to or needed to, you could do a for each loop in here. It's going to work in exactly the same way. So now what we can do is inside of here,
04:34
it's right over each of them posts and we now get a post for each of these. And then we can say this info and post ID. Or in this case, we would do something more useful inside of here. OK, let's go ahead and run this and see what happens.
04:50
So there we go. What's happening now is for every thousand we're chunking through these. We are outputting the ID of the post or doing something with it. And then it's just clearing this out of memory until we get to the point where we
05:03
have gone through all of these batches of posts and done something with them. So you can see here that we've gone all the way down to two hundred thousand pretty effortlessly. And of course, more importantly, without running out of memory.
15 episodes•1 hr 9 mins

Overview

Let's keep our Laravel applications feeling snappy! In this course, we cover the absolute fundamentals you need to keep in mind when building anything with Laravel.

While Laravel handles a lot for you, it's easy to fall into the trap of not considering and monitoring performance as you go. Keep these tips in your toolbelt, and you'll be able to develop faster apps, from the beginning.

Alex Garrett-Smith
Alex Garrett-Smith
Hey, I'm the founder of Codecourse!

Comments

No comments, yet. Be the first to leave a comment.