Amazon S3, is AWS's blob storage solution offering highly scalable storage with 99.999999999% durability.
- Buckets are the containers of storage. They're granted DNS names and are accessible over HTTP.
- Objects are stored files.
- Long-term object storage (e.g. media files) fronted by a delivery service.
- Backing storage for other services (e.g. EBS).
- Static file or website hosting.
- Read-after-write for newly created objects.
- Eventual consistency for put and delete operations on existing objects.
Objects can be shared by generating ephemeral pre-signed URLs, using that user's security credentials. The URLs encapsulate:
- the generating user's security credentials;
- a bucket name;
- an object key;
- the permitted HTTP verb; and
- expiration time.
Encryption can be performed either client-side or server-side (on S3). Note that in cross-region replication objects will be encrypted with the same keys in the source and destination: the destination's encryption configuration will not apply. The supported encryption schemes are:
- Disabled, to store items in cleartext.
- SSE-KMS, using S3-managed keys.
- AWS-KMS, using a KMS-managed key.
Storage classes determine performance and can be used as a cost management measure.
- S3 Standard is the default and is suitable for frequently accessed data.
- S3 Intelligent-Tiering allows S3 to move data between frequent and infrequent tiers based on usage.
- S3 Standard-IA is used for infrequently accessed data with standard resilience.
- S3 One Zone-IA is used for infrequently accessed data with reduced resilience, storing objects in just one AZ.
Current best practice is to use bucket policy to secure the bucket and ACLs to gate access to objects, avoiding use of ACLs to govern access to the bucket itself. The least permissive policy wins in the event of an overlap.
Note that the role's applied IAM policy also applies; ensure permissions are granted in both policies.
Bucket policy allows specifying which actions are allowed or denied for specific security principals. They're assigned at the bucket level.
An ACL can be used to control access at both the bucket and object levels. The following types of rule exist:
- List objects
- Write objects
- Read bucket permissions
- Write bucket permissions
- Read object
- Read object permissions
- Write object
Lifecycle policies allow objects in a bucket to be transitioned (or expired/deleted) based on customer-set criteria based on object attributes such as tags. Some common use cases include:
- Pruning objects older than a specified age.
- Retaining a number of previous versions of objects based on upload date.
Versioning allows a bucket to retain historical versions of files. These different versions are accessible by specifying the
versionId parameter when requesting an object.
Object deletion creates a deletion marker from which the object can be restored. The deletion marker can be deleted in order to fully purge the object version. overwriting an object creates a new version with a new
Cross-region replication can be configured between two buckets. When this is configured note that objects will be encrypted with the same keys as are used on the source, regardless of whether this configuration matches the destination bucket's configuration.
Transfer acceleration allows increasing transfer performance AWS edge locations closer to clients.
- API Gateway (public)
- Athena (public)
- CloudFormation (public)
- CloudFront (public)
- CloudSearch (public)
- CodeBuild (public)
- CodePipeline (public)
- Config (public)
- DocumentDB (public)
- GuardDuty (public)
- Macie (public)
- Neptune (public)
- Amazon Redshift (public)
- VPC (public)
- Data Analytics (public)
- Data Firehose (public)