Another form of log-watching, this time the Audit logs that Aurora can push to Cloudwatch. It's not all stuff by any means; heck, the example they use is just about failed logins.
In this case, we're reading the Cloudwatch logs for our RDS clusters.
Since Cloudwatch offers filters, in this example we're looking for QUERY commands, since we've turned Server Audit on, and are uploading connects + queries to Cloudwatch. (More information on that in: https://aws.amazon.com/blogs/database/monitoring-amazon-aurora-audit-events-with-amazon-cloudwatch/). Remember to set a cap on it, though then interesting things could be buried.
Why are we doing this? In our case, this gives us a few bits of cool info. Specifically, we can log all the DDL commands that come across our cluster, making sure everything is behaving as expected.
Showing posts with label CloudWatch. Show all posts
Showing posts with label CloudWatch. Show all posts
Monday, May 7, 2018
Tuesday, March 13, 2018
[AWS] Querying CloudWatch logs using powershell to find DDL changes in your Aurora instance.
Did I get enough buzzwords in that post title?
So, I've been working on an AWS Aurora MySQL project lately (yes, you need to qualify Aurora, because these days there's a Postgres variant).
One problem we had was getting info out of the logs. Querying the logs is possible from within MySQL, but time consuming. Ditto querying the logs that are being saved to disk. We used the AWS directions (https://aws.amazon.com/blogs/database/monitoring-amazon-aurora-audit-events-with-amazon-cloudwatch/) to get our logs to dump into CloudWatch, which offers a Powershell cmdlet to filter them (although it limits to 1mb at a time, hence the DO WHILE with a low rowcount). Unfortunately, I didn't find any actual working examples.
So while this may not be the best, it does work.
NOTE: results currently go to an out-gridview window. Obviously, change that to what you need, or:
* remove the write-host
* remove the ogv line
* wrapper the entire DO WHILE into a variable then return that at the end. BE CAREFUL DOING THIS. I tried it with a decent-sized results set (30k) and got a weird error back:
Get-CWLFilteredLogEvent : Error unmarshalling response back from AWS. Request ID:
So, I've been working on an AWS Aurora MySQL project lately (yes, you need to qualify Aurora, because these days there's a Postgres variant).
One problem we had was getting info out of the logs. Querying the logs is possible from within MySQL, but time consuming. Ditto querying the logs that are being saved to disk. We used the AWS directions (https://aws.amazon.com/blogs/database/monitoring-amazon-aurora-audit-events-with-amazon-cloudwatch/) to get our logs to dump into CloudWatch, which offers a Powershell cmdlet to filter them (although it limits to 1mb at a time, hence the DO WHILE with a low rowcount). Unfortunately, I didn't find any actual working examples.
So while this may not be the best, it does work.
NOTE: results currently go to an out-gridview window. Obviously, change that to what you need, or:
* remove the write-host
* remove the ogv line
* wrapper the entire DO WHILE into a variable then return that at the end. BE CAREFUL DOING THIS. I tried it with a decent-sized results set (30k) and got a weird error back:
Get-CWLFilteredLogEvent : Error unmarshalling response back from AWS. Request ID:
Subscribe to:
Posts (Atom)