GCP Security best practice: Scalable Architecture for Logging and Monitoring

This Google Cloud Platform security best practice is part of the Logging and Monitoring security domain.

The solution shall support scaling of the ingest, index, and search layer based on data ingest and usage profiles. The scalability model shall be refined enough to add capacity in hours.

1 – GCP has some native search/filter capabilities in Stackdriver – per project. Stackdriver Logging is a fully managed service in GCP that can ingest application and system log data from VMs, and analyze log data in real time.
Customers can collect data via Stackdriver agents that can scale as needed:

  • Admin console logs – Admin console audit logs, user audit logs, Separate API and UI logs
  • GCP console audit logs – Admin activity logs
    (always enabled); Data access logs (disabled by default)
  • VMs running Stackdriver agent – common third-party applications, system software
  • Network logs – VPC Flow logs, Cloud CDN logs

2 – You can export to BigQuery for additional analysis

3- Or PubSub and use connectors that integrate with elastic search