Conor Hunt’s Tech Blog

There aren’t too many services that I pay money for on the Internet. It is a web 2.0 world - there is a lot of free stuff out there. For me to pay for an app it has to provide pretty significant value. So here is the list of apps I pay for. Let me know of any apps that you think are worth paying for, or any alternatives to these?

Sneakemail - $12 a year - Disposable email addresses

Probably the best bargain of the bunch. This helps keep my email free from spam. Every time I sign up for a new site I create a disposable address through sneakemail. It can be a completely randomly generated address 1afasfasd12@sneakemail.com or it can be an address associated with a key you choose - someaddress-yourkey@sneakemail.com. You can even reply through your anonymous sneakemail address without revealing your real email address. There are a ton of advanced options for filtering etc.

Mozy - $5 a month - Backup service.

I posted about Mozy previously and I’ve been using it for quite a while now. The latest version of the Mac client is 0.9 and it appears to be rock solid. It works away in the background backing up my files. 20 gigs of my documents, code, photos and videos are now safe in some data center somewhere. Well worth the $5 a month - you don’t even have to think about it once it is up and running.

Slicehost.com- $20 a month - VPS hosting service.

I think everyone and their dog has a slice these days. It is an excellent VPS hosting service. I’ve had mine for over a year now and it has been running great. It is the basic 256 meg slice and I run a bunch of Rails/PHP/Static sites on it using the litespeed web server to keep memory usage down.

Namecheap.com - $9.95 per domain per year - Domain registrar.

I gave up on godaddy a while ago becoming sick of their horrible interface with the incredibly annoying upsells. Namecheap has some upselling, but the interface is much cleaner, faster and not nearly as intrusive. I have several domains registered with them.

Trustfax.com - $30.00 per year - Internet fax sending and receiving service.

I’m not sure that I’m getting my money’s worth from this one, but they provide me with a dedicated toll-free phone number that emails me my faxes. I can even send faxes through them. I’ve had it for a few years and it works great, but at this point I get so few faxes that I think I’ll probably cancel it.

Google Storage - $20 per year - 10GB extra storage for picasa web and other google apps.

I use this for my photos stored in picasaweb. I’m not sure that it is worth it, there seem to be plenty of other photo web viewing apps out there offering free storage and good interfaces. I might switch to one of them at some point.

StreetEasy Insider Account - $10 a month - Recorded sales for NYC. Searchable, organized by building and up to the minute.

Ok fine. I don’t pay for this one since I work for StreetEasy, but I would if I was looking to buy an apartment in NYC ;)

Ick. I really don’t like the idea of Google Knols. Basically “authoritative” pages on a particular topic authored by a single user (see this example), with the opportunity for revenue sharing through Google ads placed on the page. Why would Google redo what Wikipedia already has - informative encyclopedia like content - but without all of the features that make Wikipedia great?

• You don’t get paid for editing a Wikipedia page - There is no motivation to bias your page towards what will earn you the most money. The most popular page isn’t necessarily the most accurate.
• Anyone can edit a Wikipedia page - Lots of downsides, but it works with enough eyes. In the long term not having this is very dangerous. What happens if the page owner decides to slip in an ad for a company, or decides to stop updating it? Wikipedia pages can correct this quickly, but in a Knol page it is likely to persist for a while once the page becomes popular and has a high ranking.
• Wikipedia pages are anonymous - Therefore there is no reason to create duplicate content for your own personal glory. Everyone has to work together to create accurate and neutral pages.

We’ll see what the implementation details are, but I’m worried that it will steal contributors from Wikipedia and create a mess of content that competes for popularity instead of accuracy and neutrality.

After setting up gitweb today on my litespeed webserver I started to wonder why it doesn’t get more press as a web server, and especially as a webserver for Rails applications. After having used nginx, lighttpd, Apache with Mongrel and FastCGI to deploy Rails apps I find litespeed to be preferable to all of them; far easier to manage, just as fast, and just as scalable for my usage. Perhaps it is not as popular because it is not open source, this is definitely an issue, but for me this is outweighed by the ease of administration and the the fact that there is a free version. After having used it for over a year here are some of the things I like (with screenshots where I could):

Great Admin Interface - You can monitor the server usage in real time in great detail. Looking at ram usage, log files, bandwidth, requests per second etc. All in realtime and with nice realtime graphs. Setup of virtual hosts and other server config is just as easy. You don’t have to use the admin interface, you can also use text files.

Simple Upgrading - When a new version is available it pops up in the admin console. You click the link and it asks you if you want to upgrade. It downloads the upgrade and does it all. If you need to roll back to a previous version it keeps those around and let’s you rollback just as easily.

Rails Apps - Litespeed has built in support for Rails and apps are incredibly quick to get up and running. It uses a custom FCGI replacement called LSAPI and it also manages Rails processes dynamically (like dynamic FCGI processes.) This is excellent. I have a slicehost instance with 256 megs and I have a several Rails apps running on it. Instead of having to have many memory hogging rails processes running, each not getting much usage, litespeed fires processes up as necessary based on requests.

On and off I have been looking for a cost effective offsite backup system for years. I was never able to find anything reasonably priced for the amount of storage I needed. However as hard drice prices have dropped a new batch of online remote backup services has appeared with much more reasonable costs. One of these that caught my eye was Mozy, advertising unlimited storage for $5 a month. My backup needs are probably around 15 gigs, so this was an attractive price compared to the other services.

Unfortunately the Mac 0.4 beta got bad reviews, so I decided to pass. The next version, 0.5, however got better reviews and so I downloaded it to try it out. The build that I downloaded is version 0.5.1.0-215 and is 6.2MB. To get to the download I had to first set up a Mozy account and answer some quick demographic questions. This was painless except that Mozy wouldn’t let me use my discardable @sneakemail.com address that I use for most signups!? So they got another spam email instead.

The install was simple. Run an installer package which then prompts you to run the backup wizard. The backup wizard takes your account information, an encryption password if you want, and then scans your system to figure out common files you might want to backup. This includes, iPhoto, iTunes, Documets, iCal, Mail, Documents and so on. Very simple and, for the non power user, very straightforward. You can also create other backup ’sets’ where you select other directories or files and add them to a named set to be backed up, or you can select individual files and directories to be backed up. I created two sets of old documents as a test, I didn’t have enough room to back everything up since I was testing the free 2 gig version of Mozy.

Backup speed was pretty fast. It took about 28 minutes, including upload time, to backup 119 megabytes of data. According to reviews previous versions of Mozy were slow, but my version was fast and looking at activity monitor it took very little CPU and very little memory while it was doing its job. The upload bitrate averaged around 650Kb/s (~81 KBytes/sec)

Restoring from a backup set was straightforward. I clicked the mozy menu (Mozy Menu Screenshot), selected ‘restore files’, selected the backup date I wanted to restore from, selected the directory I wanted to restore and then restored it to my desktop. It wasn’t incredibly fast, taking about 4 minutes to restore 10 megs of data, but fast enough and without any problems. They also have a web restore option where you can download files from your web browser, and a DVD restore option where you can have DVDs of your backups shipped to you. The DVD option seems really expensive though - $0.50/GB + $30 processing + $35 shipping.

So far from my initial testing I can reccomend Mozy. I’m going to sign up for their $5 a month unlimited plan and see how that goes.

Vineet asked me to write a ruby Daylife API client (DayPI), so I created a quick version that can consume their API. Daylife is an excellent news aggregation site that is just down the road from StreetEasy in New York. I highly reccomend checking them out, I use them all the time. You will need an API key from http://developer.daylife.com to use the API and the docs should be available there too.

The ruby version is pretty simple, and as of now hasn’t gone through much testing, but feel free to contact me if you have any questions.

Ruby Daylife API Library

Example Usage:

require 'daylife'
a = Daylife::API.new('your api key','sharedsecret')
r = a.execute('search','getRelatedArticles', :query => 'sam adams', :limit => 5)
if(r.success?)
 r.articles.each {|a| puts a.headline }
 puts r.articles[1].source.daylife_url
else
  # output the error message
  puts "Error: #{r.code}: #{r.message}"
end

The Gotham Ruby Conference was this Saturday at the Google NYC offices (my company was a sponsor.) I really enjoyed the conference, the Ruby community in NYC isn’t all that large, but there are some very smart people in it and there were some great talks. As usual though some of the best discussions were in the hallway or in the backchannel on IRC. There is a good overview of the talks on JP Reardon’s blog and Bryan Helmkamp live blogged notes from the talks.

Some of my observations:

1. NET programmers and Ruby programmers apparently move in completely different circles. There were no mentions of .NET at all. JRuby was discussed and Java was at times referred to as “the COBOL of the web era,” but .NET gets no love.
2. I have newfound respect for Amazon S3 and EC2. One of the most interesting hallway discussions I had was with a company heavily using S3 to scale their site, and using EC2 to bring up testing environments. We’ve had discussions about this at work and we’d use EC2 except that not having persistent storage apart from S3 won’t work for us. One of the solutions suggested was to bring up an EC2 instance once a week and sync it with your MySQL DB late at night and then write this image back to S3. I’m going to be looking into that.
3. Trotter’s presentation on testing rails apps was great. I rewrote a complicated system at work recently and I highly regret not writing the tests first. I wasted a lot of time because I didn’t have an automated code/test/code cycle from the start.
4. Stephen Becker IV presented a system to add comments to RDoc documentation during the lightning talks. Just adding to the motivation to get back and finish the next version of Rannotate. Hopefully we’re going to be collaborating and he has already sent me an email with questions on getting Rannotate up and running. Sweet! Stephen is also an engineer for Vonage and having to tell him that I had worked in Verizon VoIP made me somewhat ashamed. I’m not proud of any connections, however distant they are, to the current lawsuit.
5. Four free drinks per person at the after party + lots of extra drink tickets made for a lot of drunk Rubyists. It was a good time.

Recently I ran into the issue that ‘breakpoint’ stopped working with ruby 1.8.5+. There are some workarounds to make it work again, but while looking for them I found a number of references to ruby-debug, an alternative. I’m glad that breakpoint broke (haha) because ruby-debug is an excellent and much more advanced replacement.

Setting up and using Ruby-Debug is very simple:

1. gem install ruby-debug
2. require ‘ruby-debug’ and then Debugger.start (for Rails apps put this in your environment.rb)
3. then put a debug statement wherever you want to break into the debugger.

There are several different ways of using ruby-debug, but I mostly use it as an advanced version of breakpoint. I place debug statements in my app where I am having issues and use the ruby-debug command line to print variables and step through lines of code.

The main commands I use are:
pp var - pretty prints a variable currently in scope
n - step to the next line of code
cont - continue until the next ‘debugger’ statement
irb - break into an irb session

A better and more complete tutorial is available from Kent Sibilev, the author of ruby-debug.

I haven’t noticed any performance issues with my apps when I run with ruby-debug. However I have noticed a few small issues. Sometimes it outputs messages to stdout about recycled objects, not sure what this is about, but it doesn’t seem to affect the running of the application. Another more important issue is that it is sometimes unable to debug an application and will throw errors at your debug points. You can fix that, at the expense of some performance, by setting Debugger.keep_frame_binding = true. More on that error here.

I’m sure this will not be of interest to many people, but I had been looking at private file sharing networks and came across this FileSharingParagraph written by Why. I was interested in how it worked so I spent a little time de-obfuscating and commenting the code. The result is here:

http://www.conorhunt.com/code/whys_p2p.rb

I learned a few new things about Ruby syntax from de-obfuscating this.

1. I didn’t know that the ruby -s command allowed you to address command line parameters passed in with $. Ex ruby -s test.rb -S=”test” would then set up $S == test.
2. I also didn’t know that you could get a section of a string with a regex using a = some_string[/regex(capture)/,capture_group].

My co-worker and I are now the proud owners of geekpens.com it will take a day or so to update, but now we have it :)

At work we use Subversion for our source control and we have implemented a few useful checks in an SVN hook to prevent us (well mostly me) from doing stupid checkins. One of the checks makes sure that you haven’t checked in any code with a breakpoint in it. I use breakpoints for local debugging and it is a pain in the ass when you check one in and then it causes your remote environment to freeze. The other check makes sure there aren’t pre-existing migrations with the same version number as one you are trying to check in. Very useful when you have a lot of people checking in migrations.

The full code is here. To have it run place it in your svnroot/hooks directory in a file called pre-commit and make sure that file is executable (oh and it is in ruby, so you will need ruby too.) You can read more about SVN hooks in the Hook Scripts section of the svnbook documentation, but here is a short description…

The pre-commit file, as the name suggests, gets run before every commit and if it exits with a non-zero value it will prevent the commit from happening and send anything written to STDERR back to the client.

Subversion passes in a few arguments when it calls the file. The first is the path of the repository and the second is the id of the transaction (before commit each transaction gets a unique id.) Using this transaction id you can then call a command line tool called svnlook and get information about the current transaction; what is getting checked in, log messages etc.

Here is a short piece of code that makes sure every checkin has a log message associated with it.

#!/usr/bin/env ruby
repo_path = ARGV[0]
transaction = ARGV[1]
svnlook = '/usr/bin/svnlook'

commit_log =  `#{svnlook} log #{repo_path} -t #{transaction}`

if(commit_log.nil? or commit_log.strip.size == 0)
  STDERR.puts("No log message associated with this check-in, please enter a log message")
  exit(1)
end