" /> BP's Weblog: August 2006 Archives

« July 2006 | Main | September 2006 »

August 23, 2006

So... close!

Well, my scooter is sitting just shy of 5000 miles. Tomorrow would have been(/might still be) Piaggio 5k day.

However, something's wrong. So, I got to walk this route home tonight (astute readers will notice that it ends not at my home - I managed to get the thing to start again within view of my block, but short of it). Only 1.4 miles... except, pushing a 200lb scooter. sigh

All the same, 5000 miles in 19 months is a lot of diverted car miles.

Technorati Tags:

August 17, 2006

Fun with torrents, and Amazon S3

I've been watching Amazon's S3 service since it was first announced (the land grab is over, BTW - S3 announced a revision which allows you to use your own domain name in the hosting). I'm generally a fan of services priced pay-as-you go, especially when they're done with good technology following best-practices. S3 does all of these things. And it's about as cheap as reliable bandwidth, storage, and scale can come these days, too.

But, since I don't have any Web 2.0 startup ideas, nor any large files to distribute, I haven't gotten to play around with S3 too heavily. That all change this morning, when a new Nerd Vittles tutorial went up (check it out here). This was perfect - there was no torrent download, and a plea for help (in the form of a "downloads only available while the $ for bandwidth remain plea). As the post covers VoIP with open source tools, Virtual Machines, and a chance to tinker with S3 - well, most who know me can guess I was salivating at the thought of putting all three together.

And, that's what I did. I downloaded the original file, uploaded it to a bucket I have on S3. Then, I copied the generated S3 .torrent to my own server, and told the Nerd Vittles folks where to find it. Meanwhile, I had copied the original file up to my hosted server, and started up a seed there, as well (this saved me having to pay for an extra download from S3).

So, now I had Amazon's super-reliable service providing a seed, a tracker, and there was now a bittorrent flashmob forming. I could augment S3's seed with one of my own servers and whatever other bandwidth I had around, but I could also sit back and know that S3 would keep things alive, no matter what else I did.

S3 actually provides a pretty reliable backing store - it appears to provide about 75-100 Kb/s to each peer that shows up. This can add up pretty quickly, at least, if there are enough people in the mob to keep things moving, and you have (relatively) free other bandwidth to contribute. So, not content just to watch people transfer the torrent and slowly tick up my S3 bill (the entire project has cost a little over $1 so far today), I had to experiment with one more variable. Objects in S3 have an ACL associated with them. According to the docs, you have to make an object "public-read" for it to become available as a torrent source. This is true, if only because you can't get the initial .torrent created otherwise.

However, it occurred to me S3's tracker might, possibly, continue running, even if you change the ACL so that the seeder would have to drop out of the swarm. So, I changed the ACL on the original S3 object to remove "public-read", and waited. To check that things were still swarming along nicely, I even started a download on another machine. Happily, the S3 tracker is still playing the tracker role, but the S3 seeder has stopped racking up bandwidth. Since I'm still running another seed on my own machines, I can be sure that the swarm will stay healthy, but I can also pay Amazon only for the super-reliable tracker infrastructure, and the (modest) cost of storing an inert copy of the file.

Now, I just gotta' point a directional antenna at the newly-launched Google WiFi, which I can't currently get indoors, and bump up the seeding a bit more. That is, assuming Google wasn't savvy enough to limit bittorrent bandwidth on its WiFi. More on that if I get a chance to test it.

Some conclusions:

  1. Host the .torrent yourself, rather than using S3's url?torrent REST trick. This means you can continue to distribute the .torrent file even if you knock the S3 seeder out of the swarm.

  2. Hosting the .torrent also means that it's difficult (impossible?) for leachers to find the original URL on S3. Otherwise, if you link directly to the torrent URL on S3, it is possible for savvy users to just use normal HTTP download, and make you foot the bill for the entire download (S3's http is very fast - this would be a tempting trick, depending on the state of the BT swarm).

  3. If you have other seeds, or trust that your swarm is well-enough established to keep running on its own, you can remove the "public-read" ACL and still have S3 host the tracker on its reliable infrastructure. There are several open requests with Amazon to provide tools for how to manage bittorrent usage. So far, this is the only tool that I know of that helps, and it's not exactly automatic.

Technorati Tags: , ,

August 03, 2006

Wifi, wifi, everywhere!

As many of you might know, I recently moved to Mountain View. That turns out to be good timing, since Google is in the midst of preparing Google Wifi for launch. Actually, I've had a chance to try it. So far so good, although, it really highlights the difference in Wifi reception between my old 800Mhz iBook and my 12" Powerbook...

Meanwhile, I caught a mention yesterday on GigaOm of a very local startup building mesh networking hardware. I've always found the idea of self-meshing networks intriguing, both from a "kick the ISP out of the loop" long-term perspective, and for various short term reasons ("why can't I just add more APs to make my network coverage better" being prime). Looking them up, it turns out they're close enough to where I just moved that there's even a chance of hitting them with a high-gain antenna. So, I contacted them - and, well, there's a chance we might try to set up a mesh to cover the distance. We'll see. It might turn out to be handy to have a sibling that lives about half way in between, too. Meraki's $50 meshing wifi router does have impressive specs, especially for its price. I wonder how long it would run on one of my old 12v electric scooter batteries?

Technorati Tags: ,