Blog
It Was Fast With Ten Rows
- software-engineering
- php
- laravel
- typescript
- performance
The feature works. The demo is quick. QA signs off. Then it ships, the table grows from ten rows to ten thousand, and the page takes six seconds to load.
Nobody changed the code. The data changed. And the code was quietly doing way more work than anybody noticed.
The loop that runs a query
Here's the classic, in Blade:
@foreach ($posts as $post)
<h2>{{ $post->title }}</h2>
<p>By {{ $post->author->name }}</p>
@endforeachLooks harmless. But $post->author is a lazy-loaded relationship. If it isn't loaded yet, Eloquent runs a query to get it. One query for the posts, then one more for every post. Fifty posts, fifty-one queries.
That's the N+1 problem. It has a name because everybody hits it.
The fix is to tell Eloquent up front:
$posts = Post::with('author')->latest()->paginate(50);Now it's two queries no matter how many posts. One for the posts, one whereIn for the authors.
Make lazy loading yell at you
Better than remembering is having the framework catch it. In AppServiceProvider:
use Illuminate\Database\Eloquent\Model;
public function boot(): void
{
Model::preventLazyLoading(! $this->app->isProduction());
}In local and testing, a lazy load now throws an exception instead of silently running another query. In production it stays off, so a missed relationship is a slow page, not a broken one.
Turn this on in a legacy app and you'll probably find a few. Good. That's the point.
Counting doesn't need the rows
Another one that hides in plain sight:
{{ $post->comments->count() }} commentsThat loads every comment for every post into memory just to count them. Ask the database for the number instead:
$posts = Post::with('author')->withCount('comments')->paginate(50);Then use $post->comments_count. There's withSum and friends too, for the same reason.
The frontend does it too
N+1 isn't just a database thing. It shows up in the browser as a pile of HTTP requests:
const authors = await Promise.all(
posts.map((post) =>
fetch(`/api/users/${post.authorId}`).then((res) => res.json())
)
);Promise.all makes it feel fast because the requests run in parallel. It's still fifty requests, fifty trips through middleware, auth, and the database. And with real data, it's often the same handful of authors fetched over and over.
Batch it. Dedupe the ids and ask once:
const ids = [...new Set(posts.map((post) => post.authorId))];
const res = await fetch(`/api/users?ids=${ids.join(",")}`);
const authors: User[] = await res.json();
const authorsById = new Map(authors.map((author) => [author.id, author]));The Laravel side is one query. Clean the input and cap it so nobody sends ten thousand ids:
$ids = collect(explode(',', $request->query('ids', '')))
->map(fn ($id) => (int) $id)
->filter()
->unique()
->take(100);
return User::whereIn('id', $ids)->get(['id', 'name']);Even better, if the frontend always needs the author, have the posts endpoint include it. Then there's nothing to batch.
Count your queries
You can't fix what you don't see. Laravel Debugbar and Telescope both show the query count per request. If you'd rather not install anything:
DB::listen(function ($query) {
logger()->debug($query->sql, ['ms' => $query->time]);
});Put it in a test, too, so it stays fixed:
Post::factory()->count(30)->create();
DB::enableQueryLog();
$this->get('/posts')->assertOk();
$this->assertLessThan(10, count(DB::getQueryLog()));Seed more than a couple of rows first, or the test proves nothing. That test doesn't care how fast your laptop is. It cares whether someone added a query inside a loop, which is the thing that falls over at scale.
Test with real-sized data
Ten rows hide everything. Missing indexes, N+1 queries, whole tables loaded into memory. Seed something closer to production:
Post::factory()->count(5000)->create();Then run EXPLAIN on the queries that matter. If a query filtering by user_id and sorting by created_at is scanning the whole table, it needs an index:
$table->index(['user_id', 'created_at']);Don't load what you won't use
A batch job that starts with Post::all() on a big table loads every row into memory at once. Walk it in pages instead:
Post::where('status', 'draft')
->where('updated_at', '<', now()->subYear())
->lazyById(500)
->each(fn (Post $post) => $post->update(['status' => 'archived']));(If you don't need model events, a single update() query beats both.)
The habit
When something's slow, count before you optimize. How many queries? How many requests? How many rows came back that nobody used?
Usually the code isn't slow. It's doing the same small thing a few thousand times. It was fast with ten rows. Write it like there will be ten thousand.