Listing 12 million S3 objects in a desktop app: a real-world test

Tested 2026-10-25 · By the BucketWorx team. BucketWorx isn't released yet; this is a development build.

The setup

This is a real data set, not empty test files. File size doesn't change how fast a bucket lists, though: listing returns each object's name, size and date, never its contents. What matters is the number of objects and the length of their names. Flat view lists every object in the bucket, in every folder, as one list, so the whole bucket can be sorted and searched at once.

Results

Why listing a big bucket is hard

S3's listing API returns at most 1,000 objects per request, and each request has to wait for the one before it, so 12,020,000 objects take at least 12,020 requests in a row. Most of the total time is therefore the provider and the network, and it will vary. The app's job is to keep up with them: show rows as they arrive, stay responsive while millions of rows load, and search all of them without stalling.

Run it yourself

Because file size doesn't affect listing, you can reproduce the test with empty objects. Upload 12,020,000 zero-byte files with names like the ones above into a test bucket, then list the whole bucket in any S3 client and time it. On S3 Standard in us-east-1, the uploads cost about $60.10 in request fees ($0.005 per 1,000), and listing them once costs a few cents. Delete the bucket afterwards.