SpidrFast and easy to use web spider developed in Ruby | |
Download |
Spidr Ranking & Summary
Advertisement
- License:
- Freeware
- Price:
- FREE
- Publisher Name:
- Hal Brodigan
- Publisher web site:
- http://raingrams.rubyforge.org/
- Operating Systems:
- Mac OS X
- File Size:
- 42 KB
Spidr Tags
Spidr Description
Fast and easy to use web spider developed in Ruby Spidr is a versatile Ruby web spidering library that can spider a website, multiple domains, certain links or infinitely. Spidr is designed to be easy to use and fast. Here are some key features of "Spidr": · Custom User-Agent strings. · Custom proxy settings. · Black-list or white-list URLs based upon: Hostname, Port number, Full link, File extension · Provides call-backs for: Every visited URL, Every visited Page, Every visited URL that matches a specified pattern. Requirements: · Hpricot
Spidr Related Software