Fatest way to retrieve S3 objects metadata (NodeJS API)

I stored custom metadatas in my S3 objects (I stored a descriprition of the file in the meta x-amz-metadata-description).

I'd like to retrieve a file list with S3 objects properties, including metadatas.

I thought that s3.listObjectsV2 can help me but it seems that this method cannot retrieve meta data (see official doc here)

I saw that I can retrieve those medata with getObjects, but it will increase a lot the number of call and the latency of the request.

Do you have any idea to bulk retrieve metadatas of a list of objects ?

Thanks, Romain.


ANSWERS:


you need aws s3api head-object which does exactly what you want

The HEAD operation retrieves metadata from an object without returning the object itself. This operation is useful if you're only interested in an object's metadata. To use HEAD, you must have READ access to the object.

aws s3api head-object --bucket <mybucket> --key <value>

It is also available in nodeJS API, see

The method is called once per key, if you need to call for multiple items you need to cupple that with other method, lets say as an example you need all metadata for all your csv files from your bucket you would run

aws s3 ls --recursive <mybucket> \
| grep ".csv$" | cut -c 32- \
| xargs -I {} aws s3api head-object --bucket <mybucket> --key {}

The first part of the command list all files from your bucket, you grep only for a particular extension, strip out characters so you get only the key, and pass this key as argument of the head-object command



 MORE:


 ? Time to live for SimpleDB or DynamoDB
 ? Time to live for SimpleDB or DynamoDB
 ? Time to live for SimpleDB or DynamoDB
 ? Scaling MongoDB on EC2 or should I just switch to DynamoDB?
 ? Is Amazon SimpleDB more reliable/available than DynamoDB?
 ? Amazon SimpleDB or DynamoDB
 ? Time to live of a item in dynamodb
 ? AWS SimpleDB how to setup
 ? Sorting in Amazon SimpleDB not working
 ? Pricing differences between AWS SimpleDB and DynamoDB