AWS SAM

I’ve been struggling lately with thoughts on how to better manage my API Gateway and Lambda stuff. The serverless framework is nice, I suppose, but I want to have more control over things so I can get my hands dirty when I want to.

The AWS SAM models is pretty close to what I want, but the documentation is just awful. See for yourself. But progress has been made, so I’ll share my findings here.
Continue reading “AWS SAM”

AWS SAM

Getting Usage Info From CloudFront Logs

Say you have a bunch of sites that are hosted on an EC2 instances, and for various reasons you’ve setup CloudFront to help handle the traffic. You have your normal webserver logs to give you part of the picture, but you probably will want to dig into the CloudFront logs as well in order to get a better picture of your actual throughput.

Thankfully, this is pretty easy. And somewhat annoyingly complicated.
Continue reading “Getting Usage Info From CloudFront Logs”

Getting Usage Info From CloudFront Logs

Using S3 as a File System

Ignoring for the fact that there are very good reasons not to do this (latency, eventual consistency, etc), using S3 buckets as filesystems actually fills a niche that AWS doesn’t currently even try. Sure you could just setup an NFS instance on EC2, or even use the storage gateway service to go an off-service data store, but that is overkill for many use cases.

So here’s how I got it done.

First of all, there are some options on how to implement this. Here are the ones I tried out :
http://code.google.com/p/s3ql/
https://code.google.com/p/s3backer/
http://code.google.com/p/s3fs/wiki/FuseOverAmazon
https://github.com/tongwang/s3fs-c

I ended up choosing s3fs-c. I used s3fs at first, but the lack of other-client compatibility was a killer. I’m still not sure why this is even a problem to be solved, given that the s3fs-c fork didn’t make major changes to get that feature. Or maybe there are major changes that I didn’t notice on my, admittedly cursory, review.

So I took an S3 bucket, and EC2 instance running ubuntu 12.04, and did the following.
Continue reading “Using S3 as a File System”

Using S3 as a File System