Log Management Standard

Our standard for log collection, storage, retention, and access control across GCP Cloud Logging and AWS CloudWatch.

Log Management

Field Value
Document ID STD-012
Classification Internal
Owner CTO (interim CISO)
Effective Date April 2026
Review Cycle Annual

This doc explains how we collect, store, and control access to our logs across GCP Cloud Logging and AWS CloudWatch.

Architecture

GCP (Cloud Logging)

Log Sources → Log Router → Log Sinks (with filters) → Log Buckets
  • Log Sources: These are our GKE containers, applications, GCP audit logs, and other GCP services.
  • Log Router: This central service receives all logs.
  • Log Sinks: Sinks are configured with filters to route specific logs to the correct destination (Log Bucket).
  • Log Buckets: These are the storage containers for our logs.

Access to logs is controlled by IAM, often using filtered Log Views to grant developers access to only the logs from their specific services.

AWS (CloudWatch)

Log Sources → CloudWatch Logs → Log Groups → Log Streams
  • Log Sources: Applications and services running on AWS.
  • CloudWatch Logs: Central logging service that collects all logs.
  • Log Groups: Organize logs by service or application.
  • Log Streams: Individual sequences of log events from a specific source within a Log Group.

How to Access Logs

GCP:

  1. Go to Google Cloud Console → Logging → Logs Explorer
  2. Use the query builder to filter by resource type, severity, or service

AWS:

  1. Go to CloudWatch → Logs → Log groups
  2. Select the Log group
  3. Select a Log stream and search your logs

Log Storage & Retention

GCP

We use several log buckets with different retention periods:

Bucket Purpose Retention Period
_Required Stores Admin Activity, System Event, and Access Transparency audit logs. 400 days (default)
_Default Catches all logs not routed to another bucket. 30 days (default)
security-logs For security-sensitive logs like audit logs, firewall logs, and VPC flow logs. 1 year
app-logs For application and GKE logs. 6 months

AWS

  • Retention is configured directly on each CloudWatch Log Group.
  • Set retention periods based on the purpose of the log group (debugging, compliance, etc.).

Role-Based Access Control (RBAC)

GCP

We use a tiered access model to enforce the principle of least privilege:

Team GCP IAM Role Scope What They Can See
Developers roles/logging.viewAccessor Specific Log View Can only see logs that match their view’s filter (e.g., logs from their own service).
SRE/DevOps roles/logging.viewer Project or Folder Can view all logs in the project.
Security Team roles/logging.viewer security-logs bucket Can view security-related logs only.
Admins roles/logging.admin Project Have full control to manage sinks, buckets, retention policies, and IAM.

AWS

Team AWS IAM Policy/Actions Scope What They Can See
Developers logs:GetLogEvents, logs:FilterLogEvents Specific Log Groups Can only view logs for their service’s log groups.
SRE/DevOps CloudWatchLogsReadOnlyAccess Account Can view all logs across the account.
Admins CloudWatchLogsFullAccess Account Full control: manage log groups, retention, IAM.

Monitoring & Alerting

GCP

  • We create log-based metrics from log content (e.g., counting the number of 5xx errors).
  • We use these metrics to set up Cloud Monitoring alerts that notify the appropriate teams via Slack or PagerDuty.
  • We have specific alerts configured for critical security events, such as unauthorized API calls, IAM policy changes, or privilege escalation.

AWS

  • CloudWatch: Monitor logs via Log Groups and set up alerts as needed.