Logo Search packages:      
Sourcecode: harvestman version File versions  Download package

HarvestMan::urlqueue::HarvestManCrawlerQueue Class Reference

List of all members.


Detailed Description

This class functions as the thread safe queue
for storing url data for tracker threads 

Definition at line 46 of file urlqueue.py.


Public Member Functions

def __init__
def add_tracker
def configure
def crawl
def empty_list
def get_base_tracker
def get_base_urlobject
def get_controller
def get_last_tracker_index
def get_locked_instances
def get_num_alive_threads
def get_url_data
def increment_lock_instance
def is_blocked
def is_crawler_queue_full
def is_exit_condition
def is_fetcher_queue_full
def is_locked_up
def mainloop
def print_busy_tracker_info
def push
def recycle_thread
def remove_tracker
def stop_threads
def terminate_threads

Public Attributes

 buffer
 data_q
 url_q

Private Member Functions

def __get_num_blocked_threads
def __get_num_locked_crawler_threads
def __get_num_locked_fetcher_threads
def __kill_tracker_threads

Private Attributes

 __qsize
 _basetracker
 _baseUrlObj
 _configobj
 _controller
 _flag
 _lastblockedtime
 _lasttimestamp
 _lockedinst
 _numcrawlers
 _numfetchers
 _pushes
 _requests
 _trackerindex
 _trackers
 _waittime

The documentation for this class was generated from the following file:

Generated by  Doxygen 1.6.0   Back to index