← Blog

Security monitoring and alerting on multi-account AWS: designing it so no one is left blind

2026-11-06

Security monitoring and alerting on multi-account AWS

This approach is the result of my AWS security preparation and certification: one of the questions that made me think the most was how to design a complete monitoring and alerting strategy for an organization with many accounts.

It looks like an exam exercise, but it's exactly the problem I've seen in real life designing Landing Zones with dozens of accounts: it's not that tools are missing, it's that nobody knows which account or region was left uncovered.

Here is the design, step by step, with the reasoning behind each decision.

First, understand what you're protecting

Before turning on a single service, I ask business questions, not technical ones:

  • What workloads exist? Public web, data processing, regulated data, sandbox?
  • What sensitive data exists and where does it live?
  • Who owns each account, and who gets notified when something happens?

Three things come out of that and hold everything else up:

  • Mandatory tags: Environment, Owner, DataClassification and Criticality, enforced with Tag Policies and SCPs. Without Owner, an alert doesn't know who to reach.
  • Continuous inventory with AWS Config and Resource Explorer, and sensitive data discovery with Macie on S3.
  • An account structure with separation of duties: Management (no workloads), Log Archive (immutable logs), Security Tooling (delegated administrator for the security services) and workload accounts grouped by OU (Prod, Non-Prod, Sandbox).

The rule that matters most here: the management account runs no workloads and no security tooling. And logs live in a separate account, so whoever compromises a workload account can't delete the evidence.

The monitoring strategy, in layers

No single service sees everything. So I think in layers, each answering one question:

  • API activity — CloudTrail (organization trail): who did what, from where?
  • Configuration — AWS Config with conformance packs and an organization aggregator: what drifted from the baseline?
  • Network — VPC Flow Logs, Route 53 Resolver Query Logs, Network Firewall and WAF: is there anomalous traffic, exfiltration or malicious DNS?
  • Threats — GuardDuty (S3, EKS, malware, RDS, Lambda): are there compromised credentials, crypto-mining or command-and-control traffic?
  • Vulnerabilities — Inspector (EC2, ECR, Lambda): which CVEs do my workloads have?
  • Data — Macie: which PII or credentials are exposed?
  • Permissions — IAM Access Analyzer: is there public or cross-account access I didn't expect?
  • Posture — Security Hub (CSPM): do I meet AWS FSBP, CIS or PCI DSS?

The principle that makes it scale: auto-enable

This is the difference between a design that works on day one and one that still works a year later. Every service is enabled from the Security Tooling account as delegated administrator, with auto-enable for new accounts.

In other words: when someone creates an account in Account Factory, that account is born monitored. Nobody has to remember anything. And if something fails, an alert goes to the platform team.

My favorite acceptance criterion for this: a new account must have CloudTrail, Config, GuardDuty and Security Hub active in all enabled regions in under 30 minutes.

Aggregate without losing the evidence

With dozens of accounts and regions, going console by console is not an option. Aggregation has three pieces:

  • Security Hub as the single pane of findings (GuardDuty, Inspector, Macie, Config and Access Analyzer), with cross-region aggregation into a single home region.
  • Amazon Security Lake to normalize logs into OCSF format in S3 and query them with Athena. It's what lets you investigate backwards: "what did this identity do 30 days ago?".
  • Amazon Detective to correlate a finding with related entities when you need to dig deep.

And the piece an auditor cares about most: the Log Archive bucket, with:

  • Object Lock in compliance mode (nobody, not even root, deletes early).
  • KMS encryption.
  • A policy that only allows writes from the service.
  • CloudTrail log file integrity validation.
  • A lifecycle rule to Glacier to control cost.

Measure what matters

A dashboard with a thousand charts isn't a useful dashboard. I keep a few metrics, but ones that change decisions:

  • Findings by severity, account and service.
  • MTTD and MTTR (time to detect and time to respond).
  • % of compliant resources per standard.
  • Coverage: % of accounts and regions with each service active. If it drops below 100%, an alarm fires.
  • Age of critical vulnerabilities.

For API events that can't go unnoticed, I create metric filters on CloudTrail in CloudWatch Logs, aligned with the CIS controls:

  • Root account usage.
  • Console login without MFA.
  • Changes to IAM, SCPs or Security Groups.
  • StopLogging or DeleteDetector.

And three views, because each audience needs something different: an operational dashboard for the SOC, a posture one for account owners and an executive one with trends and compliance for leadership.

Alert without causing fatigue

Detecting well is pointless if the alert is lost in the noise. My rule is a central EventBridge bus in Security Tooling that routes by severity:

  • CRITICAL / HIGH — immediate notification (PagerDuty or Slack) and an automatic ticket.
  • MEDIUM — ticket.
  • LOW — daily report.

And the recipient comes from the Owner tag. If there is no owner, it escalates to the SOC. That's why tagging isn't an "administrative" topic: it's what gets the alert to the right person.

For noise, Security Hub automation rules suppress false positives, but with discipline: every suppression carries a justification, an author and a review date, and is versioned in the repository. A suppression without an owner is a back door.

Remediate, but carefully

Automation is powerful and dangerous in equal parts. What I automate, and how:

  • Isolate a compromised instance: quarantine Security Group and a forensic snapshot.
  • Disable exposed IAM keys and revoke sessions.
  • Block public access on S3 for unauthorized buckets.
  • Re-enable CloudTrail or GuardDuty if someone stops them.

All with SSM Automation, Lambda and Step Functions, and one golden rule: in production, high-impact actions require human approval; in non-prod they can be automatic. Config drift remediations start in "notify only" mode until validated.

Prevention is best. SCPs prevent disabling CloudTrail, GuardDuty or Config, and restrict unused regions. That way, a StopLogging attempt in a member account is denied and raises an alert.

The real challenges (and how I face them)

Design is the easy part. The hard part shows up in operations:

  • Alert fatigue: prioritize by severity and criticality, and tune every quarter.
  • Costs: CloudTrail data events only on critical resources, filtered Flow Logs in Parquet, tiered retention and AWS Budgets with an alert at 80%.
  • Accounts or regions without coverage: auto-enable, SCPs and a coverage metric with an alarm.
  • Remediation that breaks production: "notify only" first, non-prod testing, approvals and rollback.
  • No owner: mandatory tagging and a clear RACI.
  • Log integrity: separate account, Object Lock and strict KMS.

And everything, absolutely everything, as code (StackSets or Terraform with CI/CD, validated with cfn-lint and checkov), with no secrets in the repository: PagerDuty or Slack tokens live in Secrets Manager. If it isn't code, it isn't repeatable; if it isn't repeatable, it doesn't scale.

The flow, summarized

  1. An account is created and the baseline applies itself.
  2. Logs go to Log Archive and Security Lake; findings go to Security Hub.
  3. EventBridge routes by severity.
  4. The owner is notified and remediation runs automatically when it's safe.
  5. Dashboards and Detective support the analysis.
  6. Metrics and post-mortems tune rules and controls.

The sixth step is the one most often forgotten. A monitoring system is never finished: it gets reviewed, tuned and tested with exercises (GuardDuty sample findings exist for exactly that).

And you, how do you organize security in your AWS Organization? What has given you the most headaches: alert fatigue, costs or accounts that slip past the baseline? Tell us in the comments — and if this article helped, share it with whoever is putting together their next Landing Zone.

Comments

…

Comment

When you comment, we email people who already took part in this post so they can follow the conversation. We use your email only to verify you and for those notices; you can opt out from any email with one click.