OpenSearch

lp://qastaging/~opensearch/opensearch/spider

Created by Vallery Lancey on 2011-01-22 and last modified on 2011-03-23

Members of OpenSearch can upload to this branch. Log in for directions.

No branches dependent on this one.

7. By Vallery Lancey on 2011-03-23: Made the script update older entier instead of making duplicates. Bug 740637 has more details.
6. By Vallery Lancey on 2011-03-05: -Protected the script from crashing when getting a 404.
-Made script ignore the #label portion of URLs.
-Hacked protection against an odd crash on line 108 in which the while loop doesn't stop when it should.
5. By Vallery Lancey on 2011-03-05: Fixed an issue of duplicate URLs in some situations.
4. By Vallery Lancey on 2011-01-30: Replaced use of wget with urllib2 (abeit with possible odd issues). The script is probably cross-platform compatable now.
3. By Vallery Lancey on 2011-01-30: Made a few more prints happen only in debug mode.
2. By Vallery Lancey on 2011-01-30: -Consolidated index.py, links.py and keywords.py into spider.py.
-Added config.py.
-Tempfiles and DB now default to being in ~/.opensearch.
1. By Vallery Lancey on 2011-01-22: Uploaded a fairly functional search spider.

This branch contains Public information

Everyone can see this information.