larbin

little and fast Web crawler

WWW CVSWeb GITHub
  1. Package version
    larbin-2.6.3p4
  2. Maintainer
    The OpenBSD ports mailing-list

Larbin is a web crawler (also called (web) robot, spider, scooter...).
It is intended to fetch a large number of web pages to fill the database
of a search engine. With a network fast enough, Larbin should be able to
fetch more than 100 millions pages on a standard PC.