2018.09-aws-practice-test-material-release-wm (1).pdf

AWS Certified Solutions Architect - Associate

Practice Quiz + Video - Reference Material

Content Prepared By: Chandra Lingam, Cloud Wave LLC For Distribution With AWS Certification Course Only Copyright © 2018 Cloud Wave LLC. All Rights Reserved. All other registered trademarks and/or copyright material are of their respective owners

VPC • Virtual Network Dedicated to your AWS Account • Logically isolated from other virtual networks in the AWS Cloud • Launch resources such as EC2 instances in your VPC • Select your own IP Address range • Create Subnets • Configure route tables, network gateways • Support for IPv4 and IPv6 • Simple to use Copyright © 2017 Chandra Lingam

Subnet 1 172.31.0.0/20

Subnet 2 172.31.16.0/20

Subnet 3 172.31.32.0/20

Default VPC 172.31.0.0/16

Availability Zone 1

Availability Zone 2 Region AWS Cloud Copyright © 2017 Chandra Lingam

Availability Zone 3

Subnet 1 Subnet 2

Subnet 1

Subnet 3

VPC A

Subnet 2

Subnet 4 Subnet 5

Subnet 3

Default VPC Availability Zone 1

Availability Zone 2 Region AWS Cloud Copyright © 2017 Chandra Lingam

Availability Zone 3

VPC Components Component

Description

VPC

Isolated virtual network in AWS cloud

Subnet

Isolated segment of your VPC

Internet Gateway

VPC side of connection to internet

NAT Gateway

AWS managed Network Address Translation Service to make outbound internet connection from your private subnet (IPv4)

NAT Instance

Customer managed NAT (IPv4)

Egress-only Internet Gateway

IPv6 outbound internet access

Copyright © 2017 Chandra Lingam

VPC Components Component

Description

Router

Routes traffic inside VPC

Security Group

Instance level stateful firewall. Supports only Allow rules

Network Access Control List

ACLs are subnet level stateless firewall. Supports Allow and Deny rules

Copyright © 2017 Chandra Lingam

VPC Components – Connecting VPCs Component

Description

Peering Connection

Connect two VPCs and access resources with private IP address

VPC Endpoint

Access AWS resources like S3, DynamoDB without using NAT or Internet Gateway. Limit access to resources from specific VPCs

Gateway Endpoint

New name for VPC Endpoints

Interface Endpoint

New Capability powered by AWS Private Link. Setup private connections to AWS Supported Services, Services hosted by AWS Partners, Customers and Marketplace partners

Copyright © 2017 Chandra Lingam

VPC Components – Hybrid Architecture Component

Description

Internet

Suitable for Internet accessible resources

Hardware VPN Connection Secure connection between your datacenter and VPC (over internet or over direct connect)

Virtual Private Gateway

AWS side of VPN connection

Customer Gateway

Customer side of VPN connection

Direct Connect

Dedicated Private connectivity between customer on-premises network/Offices to AWS

Copyright © 2017 Chandra Lingam

VPC Peering Connection • VPC Peering connection connects two VPCs to make it into one logical network • Data Transfer between Peered VPCs are on AWS private network and never traverses internet • Address should not overlap between VPCs • Instances can communicate using private IP addresses • VPC Peering across regions supported as of Nov 2017

Copyright © 2017 Chandra Lingam

VPC Peering Connection • Only one peering connection between two VPCs – It is bi-directional • Multiple peering connections are supported from one VPC to multiple VPCs • VPCs can be part of one account or different accounts • Owner of the peer VPC needs to accept the request

Copyright © 2017 Chandra Lingam

Network • Launch instances in your virtual private cloud (VPC) • Assign your own address range

• Keep instances in public subnet – for internet accessible systems • Keep instances in private subnet – to restrict access and reduce footprint

Copyright © 2017 Chandra Lingam

Bastion Host • Bastion Host is used to access your private resources from public internet • EC2 instances in private subnet allows SSH/RDP only from Bastion Host • Bastion Host on public subnet – allows access from specific IP address range for SSH/RDP access

• Reduce attack surface by controlling access points • Harden to protect your resources • Do not place your private key in bastion host – use SSH agent forwarding for connecting to private EC2 instances • Windows Remote Desktop Gateway Copyright © 2017 Chandra Lingam

Operating Systems • Numerous Linux distributions • Amazon Linux, Red Hat, SUSE, Fedora, Ubuntu and more

• Microsoft Windows • FreeBSD - marketplace

Copyright © 2017 Chandra Lingam

Amazon Machine Image (AMI) • Amazon Machine Image provides information to launch an instance • Template for root volume: OS, application server, applications • Additional volumes that needs to be attached to the instance • Permissions on who can launch an instance • Several choices from Amazon, vendors and community • Create your own, buy, share, and sell Copyright © 2017 Chandra Lingam

Amazon Linux AMI • • • •

Amazon provided and maintained Linux image Stable, secure, high-performance environment for EC2 No additional charge Repository access to multiple versions of common packages • Updated on regular basis include latest components • Can be used to update running instances through repository

• Includes AWS packages for integration – CLI, API, AMI tools, Boto library for python, ELB tools Copyright © 2017 Chandra Lingam

Dedicated and Shared Resource • EC2 dedicates some resources of host computer to each instance: CPU, memory, instance storage • EC2 shares common resources like disk sub system and network • When shared resource is underutilized - instance can consumer higher share • When shared resource are in demand – each receives an equal share • High I/O performance instance types allocate larger portion of a shared resource • Greater or more Copyright consistent I/O performance © 2017 Chandra Lingam

Instance Families • • • • •

General Purpose (T, M) Compute Optimized (C) Memory Optimized (X, R) Storage Optimized (I, D) Accelerated Computing (P, G, F)

Choice of CPU, Memory, Storage, Network, Hardware Acceleration for your needs. Determines the hardware of the host computer used Copyright © 2017 Chandra Lingam

General Purpose (T, M) E B S

CMN

C MN

C M N Instance Size

Copyright © 2017 Chandra Lingam

C M N

E B S

C M N

General Purpose – T2 instances • Lowest cost general purpose instance type - Balance of compute, memory and network resources • T2. micro eligible for free tier • Baseline CPU performance with ability to burst • Burst is governed by CPU credits - Accrue CPU credits when idle and use it when needed • Good choice for workloads that doesn’t use full CPU but burst occasionally • Suitable for Webserver, development environments and databases Copyright © 2017 Chandra Lingam

General Purpose – M4 instances • Latest generation and provides a balance of compute, memory, network resources • Good choice for many applications • EBS optimized at no additional cost • Support for enhanced networking • M3 Instance - SSD Based instance storage for fast I/O performance • Suitable for small-mid sized databases, data processing, cluster compute, sharepoint Copyright © 2017 Chandra Lingam

Compute Optimized (C) E B S

C MN

C

E B S

M N

Instance Size

Copyright © 2017 Chandra Lingam

C

E B S

M N

C

E B S

M N

Compute Optimized – C4 • • • • •

Latency gen, highest performing processors Lowest price per compute performance in EC2 EBS optimized at no additional cost Support for enhanced networking and clustering Ability to control processor C-state and P-state configuration on large instances • C3 – SSD based instance storage • MMO gaming, Video encoding, Distributed analytics, batch processing, science and engineering use Copyright © 2017 Chandra Lingam

Memory Optimized (X,R) E B S

C MN

C

M

E B S

N

Instance Size

Copyright © 2017 Chandra Lingam

M C

E B S

N

M

C

E B S

N

Memory Optimized – X1 • • • • • •

Optimized for large scale in-memory applications Lowest price per GiB of RAM among EC2 instances Upto 1,952 GiB of instance memory SSD Instance storage EBS Optimized at no additional cost Ability to control processor C-state and P-state configuration • Certified -SAP HANA, Apache Spark, Presto, HPC apps • Smaller R4 and R3 instances available Copyright © 2017 Chandra Lingam

Accelerated Computing (P,G,F) G P U

E B S

C M N

Copyright © 2017 Chandra Lingam

G E P B U S

C M N

Instance Size

F P G A

C M N

Accelerated Computing – P2 • • • • • •

General purpose GPU compute applications High performance NVIDIA K80 GPUs GPUDirect support for GPU-GPU peer communication Enhanced networking upto 20Gbps EBS optimized at no additional cost Machine Learning, High performance databases, computational fluid dynamics, seismic analysis, rendering, genomics workloads Copyright © 2017 Chandra Lingam

Accelerated Computing – G2 • Optimized for Graphics intensive applications • High performance NVIDIA GPUs • On-board hardware decoder for multiple real-time HD streaming • Low latency frame capture and encoding – High quality interactive streaming experience • 3D application streaming, video encoding, server side graphic workload Copyright © 2017 Chandra Lingam

Accelerated Computing – F1 • Customized hardware acceleration with field programmable arrays (FPGA) -Xilinx • NVMe SSD storage • Support for Enhanced networking • Genomics research, financial analytics, real-time video processing, security, big data search and analysis

Copyright © 2017 Chandra Lingam

Storage Optimized (I,D) N C M

N

C M

SSD Copyright © 2017 Chandra Lingam

HDD Instance Size

Storage Optimized – I2 • High storage instances with SSD backed instance storage • Very high random I/O performance • High IOPS at low cost • Support for enhanced networking • NoSQL databases Cassandra, MongoDB, scale out transactional databases, cluster filesystems, data warehousing, hadoop Copyright © 2017 Chandra Lingam

Storage Optimized – D2 • Dense storage instances – 48TB of HDD local instance storage • High disk throughput • Lowest price per disk throughput • Massively parallel data warehousing, Hadoop Map Reduce, Distributed file systems, network file systems, log or data processing applications

Copyright © 2017 Chandra Lingam

Resizing Instances

C M N

C

Copyright © 2017 Chandra Lingam

E B S

M N

Resizing Instances • Resize an existing instance based on your usage – over or under utilization • Stop instance, update to new instance type, restart • Only supported for Instances with EBS root device volume. Not supported on Instance store root device volumes • Target instance type must be compatible • Virtualization Type. HVM <-> PV not allowed • 32 bit <-> 64 bit not allowed • Some instances are restricted to VPC. You cannot use in EC2-Classic Copyright © 2017 Chandra Lingam

Firewall • Security Groups – Mandatory firewall for EC2 instances • Applies to all Inbound and outbound traffic at Instance level • Stateful filters

• Network Access Control Lists (ACL) • Applies to all inbound and outbound traffic from a subnet in VPC • Stateless traffic filters

Copyright © 2017 Chandra Lingam

Storage Options • • • • •

Amazon EC2 Instance Store Amazon Elastic Block Store (EBS) Amazon Elastic File System (EFS) Amazon Simple Storage Service (S3) Figure: Storage

Copyright © 2017 Chandra Lingam

Elastic Load Balancing (ELB)

Load Balancer Types Classic Load Balancer • Target: EC2 instances • Works on EC2-Classic and VPC • Supports HTTP, HTTPS, TCP, SSL (Secure TCP) • SSL Offloading • Sticky Sessions • OSI Layer 4 (Transport), 7 (Application) Load Balancer

Classic Load Balancer Static Mapping HTTP

HTTP/HTTPS

HTTPS

HTTP/HTTPS

Classic LB TCP Secure TCP

TCP/Secure TCP

TCP/Secure TCP

EC2 EC2 EC2 EC2 EC2 EC2

Load Balancer Types Application Load Balancer • Target: EC2 instances, Containers, Private IP Addresses • Content Based Routing (route to multiple services) • Path based routing • Host based routing

• • • • •

Load balance across different ports on a EC2 instance SSL Offloading Stick Sessions Supports HTTP, HTTPS, HTTP/2, WebSocket OSI Layer 7 (Application) Load Balancer

Application Load Balancer Rule Based Mapping Target Group 1 Default HTTP/HTTPS

HTTP

HTTPS

EC2 or Containers or Private IP address

Target Group 2 Application LB

/order HTTP/HTTPS

EC2 or Containers or Private IP address

Target Group 3 /video HTTP/HTTPS

EC2 or Containers or Private IP address

Load Balancer Types Network Load Balancer • Target: EC2 instances, Containers, private IP Addresses (based on IP Protocol data) • Very high performance, handles millions of requests per second at very low latencies • Optimized for handling sudden and volatile traffic patterns • Long lived TCP Connections (websocket) • One Static IP or Elastic IP per Availability Zone • Preserves Source IP Address • OSI Layer 4 (Transport) Load Balancer

Network Load Balancer Rule Based Mapping

Static IP

Target Group 1 Port

TCP

EC2 or Containers or Private IP address

Static IP Target Group 2

TCP Port Static IP

Network LB

EC2 or Containers or Private IP address

Which Load Balancer To Use? Classic – Recommended for EC2-Classic network Application – Flexible Application management and TLS termination Network – Extreme Performance and Static IP AWS ELB FAQ

Load Balancer Types Types of Load Balancer - Comparison Table

Cross Zone Load Balancing • For fault tolerance, EC2 instances should be distributed across two or more availability zones • Cross Zone Load Balancing controls how traffic is distributed across Availability Zones and Instances in each Availability Zone

Copyright © 2017 Chandra Lingam

Enabled - Cross Zone Load Balancing • Distribute traffic evenly across EC2 instances in all Availability zones • Happens when Cross Zone load balancing is “Enabled” • Availability Zone ‘A’ has 3 instances and Availability Zone ‘B’ has 2 instances. Each instance would receive 20% of the traffic (1/5th) • Default mode in application load balancer

Copyright © 2017 Chandra Lingam

Disabled - Cross Zone Load Balancing • Distribute traffic evenly across Availability Zones • Happens when Cross Zone load balancing is “Disabled” • Two availability zones ‘A’ and ‘B’ would each receive 50% of the traffic irrespective of number of EC2 instances in each Availability Zone • May cause higher loading if one Availability Zone has fewer EC2 instances • Default mode in classic load balancer

Copyright © 2017 Chandra Lingam

IAM Concepts

Permissions

Policy

Attach Inherit

Group

User

Role

User

Copyright © 2017 Chandra Lingam

User

Resource

IAM Role

Permissions

Temporary

Role

User

EC2

AWS Service

Federation

Role has two parts: 1. Who can assume the role and 2. What permissions does a role have Copyright © 2017 Chandra Lingam

IAM Policy Types

Permissions Types

Owner

Policy

Inline (embedded)

Managed (reusable)

AWS

Copyright © 2017 Chandra Lingam

Customer

Amazon Simple Storage Service (S3) Content Prepared By: Chandra Lingam, Cotton Cola Designs LLC For Distribution With AWS Certification Course Only Copyright © 2017 Cotton Cola Designs LLC. All Rights Reserved. All other registered trademarks and/or copyright material are of their respective owners

Copyright © 2017 Chandra Lingam

Storage Class

Standard

Standard–IA

Usage

Frequently Accessed Data

Less Frequently Less Frequently Frequently Accessed Accessed Data Accessed Non Non critical data Critical Data

Rarely Accessed. Data Archiving

Durability

99.999999999%

99.999999999%

99.999999999%

99.99%

99.999999999%

Availability

99.99%

99.9%

99.5%

99.99%

N/A

Availability SLA (Service Credit)

99.9%

99%

99%

99.9%

N/A

Concurrent Facility Failure

2

2

Stored only in 1 AZ

1

2

Redundancy (Region)

Multiple devices in multiple AZ

Same as standard

Multiple devices Fewer copies in single AZ

Same as standard

First Byte Latency

Milliseconds

Milliseconds

Milliseconds

Milliseconds

4 hours

Minimum N/A Storage Duration

30 days

30 days

N/A

90 days

Minimum Size

128 KB

128 KB

STANDARD_IA

ONEZONE_IA

REDUCED_REDUNDANCY

GLACIER

x-amz-storageclass

STANDARD

One Zone-IA

Copyright © 2017 Chandra Lingam

RRS

Glacier

Example

Quantity

Standard

Standard–IA

One Zone-IA

RRS

Glacier

Storage

100 GB

$2.3 per month

$1.25 per month

$1 per month

$2.4 per month

$0.4 per month

PUT, COPY, POST, or LIST

1,000 requests

$0.005

$0.01

$0.01

$0.005

$0.05 for 1,000 archives or restores

GET and all other

1,000 requests

$0.0004

$0.001

$0.001

$0.0004

Data Retrievals, Restores

NA

NA

$0.01/GB

$0.01/GB

NA

Data Transfer IN to S3

Expedited - $0.03 per GB Standard - $0.01 per GB Bulk - $0.0025 per GB

Free

Data Transfer OUT from S3 To Same Region

Free

Different Region

$0.02/GB

CloudFront

Free

Internet

Free Up to 1GB/month. $0.09 per GB Copyright © 2017 Chandra Lingam

Standard Storage (STANDARD) • • • •

Ideal for performance sensitive use cases Frequently accessed data (Hot Data) Real time access Default storage class for objects

Standard Infrequent Access (STANDARD_IA) • • • • • • •

Performance similar to Standard Less frequently accessed (Warm Data) Real time access when needed Additional Retrieval fee Lower cost compared to Standard Long-lived data (backup or older data) Minimum size: 128KB and Duration: 30 days

Glacier (GLACIER) • Archived data (Cold data) - No real time access • Need to restore objects • Restored to RRS storage • Kept for specified number of days and removed

• Restore takes few minutes to several hours • Expedited – 1 to 5 minutes ( < 250MB object size) • Standard (default) – 3 to 5 hours • Bulk – 5 to 12 hours. Very large restores (petabytes)

• Each Glacier object uses 8 KB Standard storage and 32KB for Glacier metadata

One Zone Infrequent Access (S3 One Zone-IA) • Launched in 2018 • 20% lower cost than Standard IA • Intended for re-creatable data and for uses that does not require multi-AZ resiliency • Can be used in Life Cycle Transitions • Example Usage: Secondary backup of on-premises data, Target storage class for Cross Region Replication Introducing One Zone IA One Zone IA Comparison

Reduced Redundancy Storage (RRS) • • • •

Storage for Non critical, easily reproducible data Hot or Warm data Real time access when needed Fewer redundant copies stored when compared to Standard storage – can configure event notification when object is lost • 400 times the durability of typical hard disk drive • Cheaper when compared to Standard storage

S3 Data Protection • In-transit protection • HTTPS endpoints for AWS Services • Client side encryption

• Data at rest • S3 Server Side Encryption – S3 encrypts object when storing and decrypts when retrieving • Client Side Encryption – Encrypt data on your side and upload encrypted data to S3. Encryption process, keys, and tools are managed by client Copyright © 2017 Chandra Lingam

S3 Server Side Encryption • S3 encrypts when writing object and decrypts when reading object • For authorized users, no difference between encrypted and unencrypted object – Transparently handled by S3 • Three key management choices • S3 managed keys (SSE-S3) • Key Management Service managed keys (SSE-KMS) • Customer provided keys (SSE-C)

• AES256 Encryption Algorithm • Object Data is encrypted. Metadata is not encrypted Copyright © 2017 Chandra Lingam

S3 Access Management • User Policies (covered in IAM Lecture) • Resource Policies • Bucket Policies (covered in IAM Lecture) • Bucket Access Control List (ACL) • Object Access Control List (ACL)

• When to use ACLs?

Copyright © 2017 Chandra Lingam

S3 - Object ACL • Control permissions at object level - Permissions vary by object • If Object owner is different from bucket owner – Object ACL is the only way object owner can grant permissions • Bucket owner cannot read until given permission • Bucket owner can deny access to object

• No user level permissions. Only at account level • Grantee: Another Account or predefined S3 groups. Account can be referred by email address or Canonical ID Copyright © 2017 Chandra Lingam

S3 - Bucket ACL Only recommended use for Bucket ACL • Grant access to S3 Log Delivery Group to write S3 access logs to your bucket • Bucket ACL is the only way in which Log Delivery Group can be granted access • No user level permissions. Only at account level • Grantee: Another Account or predefined S3 groups. Account can be referred by email address or Canonical ID Copyright © 2017 Chandra Lingam

S3 - Bucket Policy • User Policy or Bucket Policy – to manage access within same AWS account • Bucket Policy • Can be used to grant cross-account access permissions for all S3 Actions (no need to use IAM role as a proxy) • Bucket ACL can also grant cross-account access but only for some S3 Actions

Copyright © 2017 Chandra Lingam

Glacier Storage Service Content Prepared By: Chandra Lingam, Cloud Wave LLC For Distribution With AWS Certification Course Only Copyright © 2018 Cloud Wave LLC. All Rights Reserved. All other registered trademarks and/or copyright material are of their respective owners

Glacier Storage Service Cold Storage Low Cost

Durable Store data for months, years and even decades IAM based Access Control

Two ways to store in Glacier S3 Glacier Storage Class Glacier Vault and Archives

S3 Glacier Storage Class Lifecycle Rules to Transition Objects to Glacier Storage Class Interact using S3 Management Console, S3 SDKs To access, first restore to a temporary S3 Object Temporary object automatically removed after specified duration

Glacier Console and API Glacier Vault – Container for storing your data Archives – Individual files that are stored in Glacier Vault

Interact using Glacier Console, API, SDKs Asynchronous jobs to inventory archives or to retrieve an archive

Glacier Console and API Jobs can take several hours to complete Glacier can publish to Simple Notification Service Topic on job completion status Vault Lock to protect a glacier vault from changes – Useful for legal, regulatory and compliance requirements Implement strict Write Once, Read Many model

Glacier - Archive Retrieval Expedited – 1 to 5 minutes. Can provision capacity to ensure immediate processing Standard – 3 to 5 hours Bulk – 5 to 12 hours. Lowest cost retrieval option and can restore large amounts of data inexpensively

Route 53 and DNS Concepts Terminology

Description

Generic Top Level Domain (TLD)

Last part of a domain name (.com, .org, .cloud).

Geographic Top Level Domain

Domains associated with geographic areas. (.uk, .fr, .io, .in)

Domain Name System (DNS)

Worldwide network of servers that maintains domain names to IP Addresses

Name Servers (NS)

Servers in DNS that respond to DNS queries

Authoritative Name Server

NS that has definitive information about one part of a domain name

Copyright © 2017 Chandra Lingam

Route 53 and DNS Concepts Terminology

Description

Hosted Zone – Route 53

A container that has information on how to route traffic for a domain (example.com) and sub domains (www.example.com, retail.example.com)

Resource Record Set

Configuration that maps domain name to resources that can process the request. Several types of resource records are supported

Time To Live (TTL)

Time in Seconds a particular Resource Record Set can be cached

Alias Resource Record Set

Route 53 specific extension to route traffic to AWS resources such ELB, S3, CloudFront and so forth – automatically tracks backend resources. TTL setting is inherited from target service. Cannot change in Route 53 Copyright © 2017 Chandra Lingam

Route 53 Routing Policy Routing Policy

Description

Simple Routing

Used when you have a single resource performing a function. For Example, one web server serving content. In Simple Routing, Route 53 simply returns the configured values for matching resource recordset

Weighted Routing

Used when you have multiple resources performing similar function and you want to route traffic to resources in proportions that you specify. For example: Several web servers serving content, A/B testing

Latency Routing

Used when you have deployed your application across multiple regions and want to route customers to resources that offer best possible latency.

Failover Routing

Active-Passive failover support. All traffic is routed to Primary endpoint (known as Active). If primary is down, then all traffic is send to Second endpoint (known as Passive).

Geolocation Routing

Used when you want to route traffic to resources in the same geography as your users. Can be used for compliance requirements. You can support a default record set to handle requests where you don’t have resources. Otherwise, Route 53 will return a “No Answer” response Copyright © 2017 Chandra Lingam

SQS Types Standard Queues – Maximum throughput, best effort ordering, and at-least-once delivery FIFO Queues – Limited throughput, exact ordering, and exactly-once processing Table: Comparison of SQS Queue Types

Copyright © 2017 Chandra Lingam

Standard Queue Concepts At-Least-Once Delivery • On rare occasions, you might receive duplicate messages with Standard Queues • Design your application to handle duplicate messages Figure: Sampling

Copyright © 2017 Chandra Lingam

FIFO Queue Concepts Exactly-Once Processing • No duplicate messages sent to receivers Deduplication • Helps you avoid sending duplicate messages during 5minute interval • Content based Deduplication ID or Producer provided Deduplication ID Copyright © 2017 Chandra Lingam

FIFO Queue Concepts Message Group ID • Ordering is preserved within a message group • Multiple message groups within a single FIFO Queue • Only one consumer can have an inflight message in a message group • Multiple consumers can access messages in different message groups – one consumer per message group • Improve throughput and latency Copyright © 2017 Chandra Lingam

Amazon Simple Notification Service Fully managed Push Notification Service Send individual messages or fan-out messages to large number of recipients Send push notifications to mobile devices Deliver messages to Amazon Simple Queue Service (SQS), AWS Lambda, HTTP(S) endpoint, Email Copyright © 2017 Chandra Lingam

SNS Concepts Topic – Logical Access Point and Communication Channel Publisher – Sends message to a topic

Subscriber – Subscribes to a Topic using variety of supported protocols and receives messages Figure - Components Copyright © 2017 Chandra Lingam

SNS Usage Scenarios Fanout – broadcast a message to multiple consumers Application & System Alerts – Alert about changes to your application or infrastructure

Push Email & Text Messaging – Transmit messages to individuals or groups via email and SMS Mobile App Push Notification – Notify directly to your App users Copyright © 2017 Chandra Lingam

Relational Database Engines – High Availability

Primary

AZ 1

Replication

Standby

AZ 2

Aurora Overview • MySQL and PostgreSQL compatible • Performance: 5X MySQL, 3X PostgreSQL • Six way data replication across 3 Azs • For Writes, four copies must be stored safely before transaction is successful

• In case of primary crash, a read replica is promoted as primary – typically under 60 seconds • Low latency read replicas (up to 15) • Support for Cross Region Replication

Aurora • Cluster Endpoint • Points to Current Primary Instance • Suitable for Writes and Reads mydbcluster.cluster-123456789012.us-east-1.rds.amazonaws.com:3306

• Reader Endpoint • Points to Read Replicas • Suitable for Reads • Multiple Read Replicas are load balanced at connection level mydbcluster.cluster-ro-123456789012.us-east-1.rds.amazonaws.com:3306

• Instance Endpoint • Points to Individual Aurora Instance

Aurora Serverless (GA as of Aug 2018) • Aurora Serverless - Suitable for use cases that are intermittent or unpredictable • Specify Minimum, Maximum Aurora Capacity Units (ACU) • 1 ACU is ~2 GB of Memory with corresponding CPU/Network • Pricing 1 ACU is $0.06 per hour + Storage + I/O • Aurora Serverless automatically scales up and down based on load • Scaling is rapid – uses a pool of warm resources

Aurora Serverless • Storage and Processing are separate – scale down to zero processing and pay only for storage • Automatic Pause and Resume – Configurable period of inactivity after which DB Cluster is Paused • Default is 5 minutes • When paused, you are charged only for Storage • Automatically Resumes when new database connections are requested

DynamoDB Core Concepts Tables Items Attributes

Primary Key Partition Key Partition Key and Sort Key

Secondary Indexes Global Secondary Index Local Secondary Index Copyright © 2017 Chandra Lingam

DynamoDB Core Concepts DynamoDB Streams • Captures data modification events in DynamoDB Tables • Ordered Set of events • Near real-time • Lifetime of 24 hours • Figure: Lambda functions to process stream events Examples: New customer – welcome email, Add new product to ElastiCache or ElasticSearch Copyright © 2017 Chandra Lingam

DynamoDB - Provisioned Throughput • • • • •

Consistent Low Latency Performance Read Capacity Units Write Capacity Units Modify any time Reduce cost – Purchase Reserved Capacity

Copyright © 2017 Chandra Lingam

Kinesis Platform • • • •

Continuous capture, store, analyze Fully Managed Scales automatically – TBs per hour Capabilities • Kinesis Streams • Kinesis Firehose • Kinesis Analytics Figure: Pipeline - Clickstream Analytics

Copyright © 2017 Chandra Lingam

Kinesis Streams Concepts • Stream is divided into Shards • Data is stored in Shards • One Shard provides: 1 MB/s WRITE, 2 MB/s READ, and up to 1,000 PUT operations • Add or remove Shards dynamically depending on need

Copyright © 2017 Chandra Lingam

Kinesis Streams Concepts • Data Record – unit of data stored in streams • Sequence Number • Partition Key • Data Blob (stored in Base64 encoding)

• Max Size per Data Record 1MB

Copyright © 2017 Chandra Lingam

Kinesis Streams Concepts • Partition Key is used to route Data Record to different Shards. • Partition Key is specified by the producer

Copyright © 2017 Chandra Lingam

Kinesis Streams Concepts • Sequence Number is an unique identifier for every data record • Assigned by Kinesis Streams • Sequence number for a partition key generally increase over time

Copyright © 2017 Chandra Lingam

ECS Architecture Different Components at Play: Figure: Architecture Figure: Scheduling Figure: Container Agent

Copyright © 2017 Chandra Lingam

ECS Terminology Term

Description

Image

Lightweight, Stand-alone, Executable Package

Container

Runtime instance of an image

Cluster

Logical grouping of EC2 Container Instances

Container Instance

EC2 instance on which the task runs on and is part of ECS Cluster

Task

One or more containers that form your application. Containers in a task are run together in the same EC2 instance

Scheduler

Responsible for placing tasks on Container Instances

Container Agent

Runs on each EC2 Container Instance. Reports current tasks, resource utilization to ECS and Starts/Stops tasks whenever it receives requests from ECS Copyright © 2017 Chandra Lingam

Terminology Term

Description

ECS Instance Role

IAM Role for the EC2 Container Instance. Need instance level permissions including access to ECS from Container Agent

ECS Task Role

IAM Role for individual Tasks. Fine grained based on task specific access needs. Remember an ECS Instance can host several different types of tasks with different access needed. Task Role helps you accomplish this.

ECS Task Execution Role ***2018***

This takes the place of instance role for Fargate Tasks. Grants permissions to ECS to pull private images from ECR, publish logs to CloudWatch logs on your task behalf.

ECS Static Port Mapping Port 80 Host Port 80

A

Port 80 Host Port 80

Port 80 Host Port 81

B Host 1

A

Port 80 Host Port 81

B Host 2

Copyright © 2017 Chandra Lingam

Host Port to Container Port mapping is hardcoded Requires coordination between container teams Limits flexibility on how many containers can be deployed Only one container of a particular image can be deployed in a host

ECS Dynamic Port Mapping Port 80 Host Port 32500

A

Port 80 Host Port 38000

Port 80

Host Port 32501

B

B Host 1

A

Auto assigned. A container port can map to completely different ports across EC2 Instance

Port 80 Host Port 37501

Port 80

Host Port 32502

Host Port to Container Port mapping is dynamic

B

Multiple containers of a particular image can be deployed in a host

Port 80 Host Port 32502

B Host 2

Copyright © 2017 Chandra Lingam

Application Load Balancer hides all this complexity from end users

ECS - Application Load Balancer – Dynamic Mapping • Containers are part of Target Group • ALB maintains Target Group mapping. For every container, it tracks (Instance ID, Port) • New Containers that are part of target group will automatically start receiving traffic

Copyright © 2017 Chandra Lingam

Fargate Serverless offering for running Containerized applications Package your application as containers Configure CPU, Memory, Networking, IAM Policies Scale very rapidly to supports tens of thousands of containers

Fargate Architecture Tasks and Scheduling Networking: awsvpc mode • Each Task has same networking properties as EC2 instance • Task gets an elastic network interface, Private IP address, Internal DNS name, optional Public IP address, security group

Fargate Pricing and Configuration Pricing • Per CPU $0.05/hr, • Per GB Memory $0.0127/hr • Per Second billing with 1 minute minimum • Time starts from image download and ends when task is terminated • Supported CPU and Memory Configuration

Comparison • Elastic Beanstalk - Easy Solution for web apps and web services • CloudFormation • Building-block Service that allows you to build and manage any AWS resource • Require you to author a template in JSON/YAML • Application deployment can be cumbersome

• OpsWorks • Powerful end-to-end solution. Scripting in Ruby • Complete application lifecycle from resource provisioning, configuration management, deployment, updates, monitoring, access control Copyright © 2017 Chandra Lingam

Elastic Beanstalk Concepts Concept

Description

Application

Logical Collection of Elastic Beanstalk components

Application Version

Labeled version of a deployable code

Environment

Resources provisioned to run a single application version

Environment Tier

Two types of environments: • Web server environment to handle http requests • Worker Environment to process SQS messages

Environment Configuration

Collection of parameters and settings to manage the resources

Configuration Template

Starting point for creating a new environment configuration Copyright © 2017 Chandra Lingam

Elastic Beanstalk Workflow Elastic Beanstalk Workflow • Create Application • Upload Code - Application Version • Launch Environment • Manage Environment

Copyright © 2017 Chandra Lingam

Elastic Beanstalk Permissions • Elastic Beanstalk Service Role • Used for AWS resource management on your behalf • Monitoring resources

• Elastic Beanstalk Instance Profile – EC2 Instance IAM Role • Used by instance to log to S3 • Upload Debug data to AWS X-Ray • …

Copyright © 2017 Chandra Lingam

Elastic Beanstalk Source Bundle • • • •

Single Zip file Single WAR file Max size 512 MB Cannot contain a parent directory in the source bundle. Subdirectories are supported

Copyright © 2017 Chandra Lingam

Elastic Beanstalk Deployment Options Deployment Option

Description

All at once

All instances are updated at the same time

Rolling

Updates are performed in batches. Old version and new version running in the environment until all instances are updated

Rolling with additional batch

Maintains full capacity by launching additional instances. When deployment completes, additional instances are terminated

Immutable

Full set of new instances for new version. Old instances are terminated after successful deployment Copyright © 2017 Chandra Lingam

Elastic Beanstalk - Blue/Green Deployment Eliminate downtime using Blue/Green Deployment

Blue - Production running old version Green - New environment running new version When Green deployment is successful, simply swap the CNAMEs of two environments using “Swap Environment URLs” option. Green now becomes the new Blue Production environment Copyright © 2017 Chandra Lingam

Elastic Beanstalk Platform Updates • AWS releases periodic updates to Elastic Beanstalk Platform • Software Component (AMIs, Tools, Elastic Beanstalk Scripts) • Configuration Component (Default settings applicable)

• Manual Update • Managed Updates - automatically upgrades to latest version during scheduled maintenance window. • Only patches and minor version updates are supported • Major version changes are not automatically applied Copyright © 2017 Chandra Lingam