Getting Usage Info From CloudFront Logs

Say you have a bunch of sites that are hosted on an EC2 instances, and for various reasons you’ve setup CloudFront to help handle the traffic. You have your normal webserver logs to give you part of the picture, but you probably will want to dig into the CloudFront logs as well in order to get a better picture of your actual throughput.

Thankfully, this is pretty easy. And somewhat annoyingly complicated.
Continue reading “Getting Usage Info From CloudFront Logs”

Getting Usage Info From CloudFront Logs

Using S3 as a File System

Ignoring for the fact that there are very good reasons not to do this (latency, eventual consistency, etc), using S3 buckets as filesystems actually fills a niche that AWS doesn’t currently even try. Sure you could just setup an NFS instance on EC2, or even use the storage gateway service to go an off-service data store, but that is overkill for many use cases.

So here’s how I got it done.

First of all, there are some options on how to implement this. Here are the ones I tried out :
http://code.google.com/p/s3ql/
https://code.google.com/p/s3backer/
http://code.google.com/p/s3fs/wiki/FuseOverAmazon
https://github.com/tongwang/s3fs-c

I ended up choosing s3fs-c. I used s3fs at first, but the lack of other-client compatibility was a killer. I’m still not sure why this is even a problem to be solved, given that the s3fs-c fork didn’t make major changes to get that feature. Or maybe there are major changes that I didn’t notice on my, admittedly cursory, review.

So I took an S3 bucket, and EC2 instance running ubuntu 12.04, and did the following.
Continue reading “Using S3 as a File System”

Using S3 as a File System

taking the joyent plunge

i’ve been happy with my AWS instances that i’m running, but decided tonight to start up a joyent instance to see how that goes. so far i’m fairly disappointed with the documentation, but it isn’t any worse than the AWS EC2 stuff.

still, the whole process was pretty easy to go through, even if it wasn’t easy to actually login to my new ubuntu VM.

i’ll be starting up a mail server on this thing to see how that goes.

taking the joyent plunge