Collect archived 4chan threads
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Leslie 9bcd4b4f0c
Actually load the configuration file
2 months ago
COPYING.txt Initial commit 2 months ago
README.md Initial commit 2 months ago
collectcrawl.conf Initial commit 2 months ago
collectcrawl.sh Actually load the configuration file 2 months ago
install.sh Initial commit 2 months ago

README.md

torako-collectcrawl

Requirements

Installation

The software can be installed via the provided install.sh script, which copies the scripts to the $HOME/bin directory.

Configuration

TBD

Usage

torako-collectcrawl should be called by cron(8) on a regular basis for every crawler node. The script makes sure that it is only running once.

    * * * * * $HOME/bin/torako-collectcrawl crawler1.example.com