CloudTrailのログを検索してみた

CloudTrailで記録されるJSONファイルはそのままでは検索がしづらい。
既にいろいろな外部サービスとの連携が発表されているが、手元で簡単に検索を行ってみた。
JSONの扱えるデータベースという事で、MongoDBを使いました。

事前準備

  1. MongoDB環境を用意 (今回はhomebrewでインストールしました $ brew install mongo)
  2. jq
  3. s3cmdもあると便利

データのインポート

まずは手元にS3からデータをダウンロードします。s3cmdのsyncが便利。

$ mkdir j3tm0t0-audit
$ s3cmd sync s3://j3tm0t0-audit j3tm0t0-audit

ディレクトリ以下の全てのgzip圧縮されたJSONファイルをインポートします。

$ cd j3tm0t0-audit
$ for x in `find . -name "*.json.gz"` ; do gzcat $x | jq .Records[] | mongoimport --collection cloudtrail --jsonArray ; done

データが入ったか、確認します。

$ mongo
MongoDB shell version: 2.4.5
connecting to: test
Server has startup warnings:
Wed Nov 20 18:37:08.665 [initandlisten]
Wed Nov 20 18:37:08.665 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
> db.cloudtrail.count()
273

入ったようです。

レコード検索

SecurityGroupに対してアクセス許可を行った履歴を検索してみます。

> db.cloudtrail.find({"eventName":"AuthorizeSecurityGroupIngress"},{"_id":0, "requestParameters":1 })
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e25f6ad2", 	"userId" : "123412341234" } ] }, 	"toPort" : -1, 	"fromPort" : -1, 	"ipProtocol" : "icmp" } ] }, "groupId" : "sg-e45f6ad4" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e25f6ad2", 	"userId" : "123412341234" } ] }, 	"toPort" : -1, 	"fromPort" : -1, 	"ipProtocol" : "icmp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	"items" : [ 	{ 	"cidrIp" : "205.251.233.160/28" } ] }, 	"groups" : { 	 }, 	"toPort" : 8443, 	"fromPort" : 8443, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	"items" : [ 	{ 	"cidrIp" : "205.251.233.32/28" } ] }, 	"groups" : { 	 }, 	"toPort" : 8443, 	"fromPort" : 8443, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e25f6ad2", 	"userId" : "123412341234" } ] }, 	"toPort" : 65535, 	"fromPort" : 0, 	"ipProtocol" : "udp" } ] }, "groupId" : "sg-e45f6ad4" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e45f6ad4", 	"userId" : "123412341234" } ] }, 	"toPort" : 65535, 	"fromPort" : 0, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e45f6ad4" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e45f6ad4", 	"userId" : "123412341234" } ] }, 	"toPort" : -1, 	"fromPort" : -1, 	"ipProtocol" : "icmp" } ] }, "groupId" : "sg-e45f6ad4" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	"items" : [ 	{ 	"cidrIp" : "205.251.234.32/28" } ] }, 	"groups" : { 	 }, 	"toPort" : 8443, 	"fromPort" : 8443, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	"items" : [ 	{ 	"cidrIp" : "0.0.0.0/0" } ] }, 	"groups" : { 	 }, 	"toPort" : 22, 	"fromPort" : 22, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	"items" : [ 	{ 	"cidrIp" : "205.251.233.176/29" } ] }, 	"groups" : { 	 }, 	"toPort" : 8443, 	"fromPort" : 8443, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e45f6ad4", 	"userId" : "123412341234" } ] }, 	"toPort" : -1, 	"fromPort" : -1, 	"ipProtocol" : "icmp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e45f6ad4", 	"userId" : "123412341234" } ] }, 	"toPort" : 65535, 	"fromPort" : 0, 	"ipProtocol" : "udp" } ] }, "groupId" : "sg-e45f6ad4" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e45f6ad4", 	"userId" : "123412341234" } ] }, 	"toPort" : 65535, 	"fromPort" : 0, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e25f6ad2", 	"userId" : "123412341234" } ] }, 	"toPort" : 65535, 	"fromPort" : 0, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	"items" : [ 	{ 	"cidrIp" : "205.251.233.48/29" } ] }, 	"groups" : { 	 }, 	"toPort" : 8443, 	"fromPort" : 8443, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e25f6ad2", 	"userId" : "123412341234" } ] }, 	"toPort" : 65535, 	"fromPort" : 0, 	"ipProtocol" : "tcp" } ] }, "groupId" : "sg-e45f6ad4" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e25f6ad2", 	"userId" : "123412341234" } ] }, 	"toPort" : 65535, 	"fromPort" : 0, 	"ipProtocol" : "udp" } ] }, "groupId" : "sg-e25f6ad2" } }
{ "requestParameters" : { "ipPermissions" : { "items" : [ 	{ 	"ipRanges" : { 	 }, 	"groups" : { 	"items" : [ 	{ 	"groupId" : "sg-e45f6ad4", 	"userId" : "123412341234" } ] }, 	"toPort" : 65535, 	"fromPort" : 0, 	"ipProtocol" : "udp" } ] }, "groupId" : "sg-e25f6ad2" } }
>

ちゃんと操作履歴が確認できました。実際には必要に応じて、SourceIPアドレスやUserAgentなども確認するとよいでしょう。