Skip to content

cloud-services.md

Latest commit

 

History

History
83 lines (73 loc) · 7.69 KB

cloud-services.md

File metadata and controls

83 lines (73 loc) · 7.69 KB

Cloud services

Context

Details

  • Configure all infrastructure using declarative code such as Terraform and CloudFormation (see everything as code).

  • Automate monitoring and alerting (see automate everything and observability.

  • Prefer serverless platform as a service (PaaS) over infrastructure as a service (IaaS) (see outsource bottom up).

  • Where not serverless use ephemeral and immutable infrastructure.

  • Engage your cloud supplier early on in the development process. They have various tools and processes to help you (e.g. AWS Well-Architected Review).

  • Make systems self-healing and resilient:

    • Be aware that terms such as "region" have different meanings across different cloud vendors
      • For example, it is not valid to compare the number of UK "regions" in AWS and Azure
      • High levels of resilience can be achieved using UK-based cloud services for providers such as AWS and Azure, if the full scope & resilience of the clouds is used
    • Also sometimes conflated in terms of resilience are cross-account and cross-region:
      • As a minimum, all systems should have a tamper-proof cross-account backup to protect against account compromise, e.g. ransomware atttack: see blueprint for AWS-based systems
      • You may wish to additionally consider cross region backups to protect against region failure
    • Be aware of the resilience of any systems on which your system depends - for example, in a region-failure scenario, a standby for your system in a second region won't help if your system relies on another system which only runs in the single region which has failed
    • Be aware of the difference between the resilience of cloud and your system's resilience in cloud
      • Understand the SLAs of the cloud services you use.
      • Every cloud service you use introduces more dependencies and more opportunities for service issues ...
      • ... but, bespoke engineering to avoid using cloud vendor services introduces additional complexity and opportunities for reliability issues
      • ... and, the risks are typically far greater for bespoke engineering, therefore: favour cloud services over bespoke engineering
    • Prefer technologies which are resilient by default: favour global-scoped (e.g. CloudFront or Front Door) or region-scoped services (e.g. S3, Lambda, Azure Functions) to availability-zone (AZ) scoped (e.g. VMs, RDS DBs) or single-instance services (e.g. EC2 instance storage).
    • For AZ-scoped services, use redundancy to create required resilience (e.g. AWS Auto Scaling Groups or Azure Scale/Availability Sets), and:
      • For stateless components use active-active configurations across AZs (e.g. running stateless containers across multiple AZs using AWS Elastic Kubernetes Service)
      • For stateful components, e.g. databases, consider use of active-active configurations across AZs (e.g. Aurora Multi-Master), but be aware of the added complexity conflict resolution for asynchronous replication can bring and potential performance impact where synchronous replication is chosen.
    • Consider use of multiple regions (e.g. for AWS eu-west-1 [Dublin] as well as eu-west-2 [London]) as a way to improve availability, though ensure data sovereignty implications are understood and accepted (see below).
    • Understand failover (e.g. RDS failover) and failed instance replacement times and engineer to accommodate these.
    • Be aware of data sovereignty implications of using any systems hosted outside the UK.
      • Make sure your information governance lead is aware and included in decision making.
      • Consider SaaS tools the team uses as well as the systems we build.

  • Services should scale automatically up and down.

    • If possible, drive scaling based on metrics which matter to users (e.g. response time), but balance this with the benefits of choosing leading indicators (e.g. CPU usage) to avoid slow scaling from impacting user experience.
    • Understand how rapidly demand can spike and ensure scaling can meet these requirements. Balance scaling needs with the desire to avoid over provisioning and use pre-warming of judiciously where required. Discuss this with the cloud provider well before go live they can assist with pre-warming processes (AWS).

  • Infrastructure should always be fully utilised (if it isn't, it's generating waste).

    • Though balance this with potential need to run with some overhead to accommodate failed instance replacement times without overloading remaining instances.

  • Keep up to date.

    • Services/components need prompt updates to dependencies where security vulnerabilities are found — even if they are not under active development.
    • Services which use deprecated or unsupported technologies should be migrated onto alternatives as a priority.

  • Understand and be able to justify vendor lock in (see outsource from the bottom up).

  • Build in governance as a side effect, e.g.

    • Segregate production and non-production workloads.

    • Infrastructure must be tagged to identity the service so that unnecessary resources don't go unnoticed (click to expand)

      AWS Config rule to identify EC2 assets not tagged with "CostCenter" and "Owner":

      {
  "ConfigRuleName": "RequiredTagsForEC2Instances",
  "Description": "Checks whether the CostCenter and Owner tags are applied to EC2 instances.",
  "Scope": {
    "ComplianceResourceTypes": [
      "AWS::EC2::Instance"
    ]
  },
  "Source": {
    "Owner": "AWS",
    "SourceIdentifier": "REQUIRED_TAGS"
  },
  "InputParameters": "{\"tag1Key\":\"CostCenter\",\"tag2Key\":\"Owner\"}"
}

      Further reading: AWS Config

      TO DO: Azure equivalent

    • Configure audit tools such as CloudTrail.