large_payloadQuery returned an excessively large dataset
The result set size is unusually large, suggesting overfetching, missing pagination, or inefficient data retrieval. Large payloads increase memory usage, serialization time, and network transfer costs.
Common Causes
- —Missing pagination on list endpoints
- —No field projection
- —Eager loading too many relations
- —No size limits on result sets
How to Fix
- 1.Add pagination — cursor-based preferred
- 2.Use field projection to select only needed fields
- 3.Implement lazy loading
- 4.Add result size limits
Example
typescript
// BAD — loads all orders including line items
const orders = await prisma.order.findMany({
include: { items: true, customer: true, payments: true }
});
// GOOD — paginate and select fields
const orders = await prisma.order.findMany({
take: 20,
cursor: { id: lastId },
select: { id: true, total: true, status: true, createdAt: true },
});Serialisation cost is invisible in query time
A query that takes 5ms can still cause a 500ms response if it returns 50,000 rows. Arsenic measures payload size separately from query duration.
Cursor-based pagination
typescript
// Offset pagination breaks on large datasets (OFFSET 10000 scans 10000 rows)
// Use cursor-based instead
// Mongoose
const PAGE_SIZE = 20;
async function getPage(lastId?: string) {
return User.find(lastId ? { _id: { $gt: lastId } } : {})
.limit(PAGE_SIZE)
.sort({ _id: 1 })
.lean();
}
// Prisma
async function getPage(cursor?: string) {
return prisma.user.findMany({
take: 20,
...(cursor ? { skip: 1, cursor: { id: cursor } } : {}),
orderBy: { id: 'asc' },
});
}