Prepperbot is a collection of programs and scripts which searches the internet for related content about and for preppers. It is written and controlled by wolfe.
Prepperbot does not ignore robot.txt files in websites.
Prepperbot’s Search engine is based on Sphider. Sphider is a lightweight web spider and search engine written in PHP, using MySQL as its back end database. It is a great tool for adding search functionality to your web site or building your custom search engine. Sphider is small, easy to set up and modify, and is used in thousands of websites across the world.
Prepperbot “finds” websites by snooping on Wolfe’s Bookmarks, and finding links within those sites.
The bot places sites into categories based on the most recent polling of the the site’s keywords. This means that site currently listed under homesteading, might be changed to some other category the next time it is scanned. Some categories maybe empty from time to time as a result as well.