Logo Search packages:      
Sourcecode: harvestman version File versions  Download package

HarvestMan::crawler::HarvestManUrlDownloader Class Reference

Inheritance diagram for HarvestMan::crawler::HarvestManUrlDownloader:

HarvestMan::crawler::HarvestManUrlFetcher HarvestMan::crawler::HarvestManUrlCrawler HarvestMan::crawler::HarvestManBaseUrlCrawler HarvestMan::crawler::HarvestManBaseUrlCrawler

List of all members.


Detailed Description

This is a mixin class which does both the jobs of crawling webpages
and download urls 

Definition at line 625 of file crawler.py.


Public Member Functions

def __init__
def __str__
def __str__
def action
def append_to_buffer
def append_to_buffer
def apply_url_priority
def crawl_url
def get_index
def get_index
def get_role
def get_role
def get_status
def get_status
def get_status_string
def get_status_string
def get_url
def get_url
def get_url_object
def get_url_object
def has_work
def has_work
def is_locked
def is_locked
def process_url
def push_buffer
def push_buffer
def receive_url
def run
def run
def set_download_flag
def set_download_flag
def set_index
def set_index
def set_role
def set_role
def set_url
def set_url
def set_url_object
def stop
def stop
def terminate
def terminate

Public Attributes

 buffer
 buffer
 links
 wp

Private Member Functions

def _initialize

Private Attributes

 _loops
 _role
 _status

The documentation for this class was generated from the following file:

Generated by  Doxygen 1.6.0   Back to index