AWS Storage Gateway is a hybrid storage service that connects on-premises applications with AWS cloud storage, addressing the challenge of integrating existing on-premises infrastructure with cloud storage services without application refactoring.
Seamlessly integrate existing on-premises infrastructure with AWS cloud storage services (like Amazon S3) without rewriting applications. Facilitates secure and efficient interaction with AWS cloud storage for applications using traditional protocols (SMB, NFS, iSCSI).
AWS Storage Gateway acts as a bridge and translator, providing a local cache and reducing the need for expensive on-premises hardware.
The service reduces costs by migrating off expensive on-premises hardware and improves backup and disaster recovery processes. It provides low-latency access to data stored in the cloud, combines infinite scalability and durability of AWS storage with local performance needs, encrypts data in transit and at rest, and provides local caching for frequently accessed data.
Storage Gateway can be deployed as a Virtual Machine, typically on local hypervisors (VMware ESXi, Microsoft Hyper-V, Linux KVM), or as a Hardware Appliance, which are physical hardware appliances available for environments without virtualized platforms.
Technical Specs: Deployment options: Virtual Machine (on VMware ESXi, Microsoft Hyper-V, Linux KVM), Hardware Appliance (physical)
The gateway presents standard storage interfaces to applications, manages data transfer to AWS, handles networking, caching, and communicates with AWS services like Amazon S3, S3 Glacier, EBS, and AWS Backup.
Hybrid storage solutions, like AWS Storage Gateway, offer flexibility by integrating on-premises and cloud storage. Cloud storage itself refers to the on-demand availability of computer storage space and data access over a network, eliminating the need for physical data centers.
AWS Storage Gateway offers several specialized types, each designed for specific on-premises storage needs and integrating with different AWS cloud storage services.
Storage Gateway supports three key storage interfaces: File, Volume, and Tape, which correspond to its various gateway types.
S3 File Gateway
The S3 File Gateway is designed for Network Attached Storage (NAS) needs, presenting file shares to on-premises users and applications. When a user saves a file, the gateway uploads it to Amazon S3 as an object, while metadata is cached locally for fast lookups. It sits on-premises and integrates with the AWS cloud via Direct Connect, Site-to-Site VPN, or the public internet, encrypting data in transit.
protocols:
NFS, SMB
connectivity:
Direct Connect, Site-to-Site VPN, or public internet
aws_storage_tiers_supported:
S3 Standard, S3 Standard-IA, S3 One Zone-IA, S3 Intelligent-Tiering. Supports S3 Glacier for archiving.
authentication:
Can integrate with Active Directory (AD)
Use Cases:
- Data migration and ingestion of on-premises file data to Amazon S3 (for data lakes, archiving)
- Offloading infrequently accessed data from on-premises NAS to S3, reducing local storage costs
- Providing a centralized, scalable, cost-effective cloud storage solution for on-premises file storage needs
- Enabling hybrid cloud workflows for analytics, machine learning, or other cloud-based processing of on-premises generated data
Amazon FSx File Gateway
The Amazon FSx File Gateway provides low-latency and efficient access to in-cloud Amazon FSx for Windows File Server from on-premises applications. It acts as a gateway to a fully managed, highly reliable FSx for Windows File Server in AWS, caching frequently accessed data locally for low-latency access.
connectivity:
Requires AWS Direct Connect or Site-to-Site VPN connectivity. Does not work with public internet.
supported_windows_capabilities:
SMB, NTFS, Active Directory authentication
Volume Gateway
The Volume Gateway provides block-level storage via the iSCSI protocol, acting like a Storage Area Network (SAN). It operates in two modes: Cache Mode and Stored Mode.
protocol:
iSCSI
cache_mode_operation:
Only frequently accessed data is kept on local cache disk, main copies in S3 as EBS snapshots.
cache_mode_on_premises_storage:
Requires lesser on-premises volumes (for hot data caching).
stored_mode_operation:
Primary data stored locally on provided disks; asynchronous, point-in-time snapshots saved as EBS snapshots in S3 for disaster recovery.
stored_mode_on_premises_storage:
Requires provisioning and configuring all local storage.
Use Cases:
- Cache Mode: Low-latency access for most used data, combined with S3 durability/cost-effectiveness.
- Stored Mode: Applications requiring extremely low latency and protection of local databases with offsite snapshots.
Tape Gateway
The Tape Gateway replaces physical tape backups for long-term archiving, often driven by compliance requirements. It eliminates physical tape libraries by using an iSCSI interface (Virtual Tape Library - VTL). Backup servers write to the virtual tape library, and data is copied over HTTPS to a tape library in Amazon S3. When a virtual tape is ejected, it can be automatically pushed to S3 Glacier or S3 Glacier Deep Archive for cost-effective long-term retention. Data is first staged in S3, it does NOT directly write to S3 Glacier or Deep Archive.
interface:
iSCSI (Virtual Tape Library - VTL)
data_copy_protocol:
HTTPS
workflow_storage_in_aws:
Virtual tapes first stored in Amazon S3, then can be automatically pushed to S3 Glacier or S3 Glacier Deep Archive upon ejection.
Use Cases:
- Replacing physical tape-based backup with a simple cloud solution
- Long-term data archival meeting compliance requirements
- Cost-effective off-site backup storage
AWS offers an alternative deployment option for environments where virtualization is not feasible.
AWS offers physical hardware appliances for branch offices without a virtualization environment.
These appliances can be ordered directly from Amazon or AWS.
They are pre-validated, optimized servers with appropriate CPU, RAM, and SSD caching for performance. Essentially, they are servers running a hypervisor to host the gateway VM.
Technical Specs: Pre-validated, optimized servers with appropriate CPU, RAM, and SSD caching
The physical hardware appliance simplifies procurement and deployment in environments where local virtualization management is not feasible.
Glossary
iSCSI
Internet Small Computer System Interface, a protocol allowing for block-level storage access over a network.
NFS
Network File System, a distributed file system protocol allowing a user on a client computer to access files over a computer network much like local storage is accessed.
SMB
Server Message Block, a network file sharing protocol that allows applications on a computer to read and write to files and to request services from server programs in a computer network.
Virtual Tape Library (VTL)
An iSCSI interface that emulates a physical tape library, allowing backup software to write data to virtual tapes that are stored in cloud storage.
EBS snapshots
Point-in-time backups of Amazon EBS volumes that are stored in Amazon S3 for durability.
Hybrid Storage
A storage solution that combines on-premises storage with cloud storage to leverage the benefits of both environments.