RubyCrawling is a project mainly written in Ruby, it's free.
Crawring Sample Code with Ruby
404: Not Found