Ausgabe der neuen DB Einträge
This commit is contained in:
parent
bad48e1627
commit
cfbbb9ee3d
2399 changed files with 843193 additions and 43 deletions
|
|
@ -0,0 +1,184 @@
|
|||
========
|
||||
queuelib
|
||||
========
|
||||
|
||||
.. image:: https://secure.travis-ci.org/scrapy/queuelib.png?branch=master
|
||||
:target: http://travis-ci.org/scrapy/queuelib
|
||||
|
||||
.. image:: https://img.shields.io/codecov/c/github/scrapy/queuelib/master.svg
|
||||
:target: http://codecov.io/github/scrapy/queuelib?branch=master
|
||||
:alt: Coverage report
|
||||
|
||||
|
||||
Queuelib is a collection of persistent (disk-based) queues for Python.
|
||||
|
||||
Queuelib goals are speed and simplicity. It was originally part of the `Scrapy
|
||||
framework`_ and stripped out on its own library.
|
||||
|
||||
Note: Queuelib isn't thread-safe.
|
||||
|
||||
Requirements
|
||||
============
|
||||
|
||||
* Python 2.7 or Python 3.3
|
||||
* no external library requirements
|
||||
|
||||
Installation
|
||||
============
|
||||
|
||||
You can install Queuelib either via the Python Package Index (PyPI) or from
|
||||
source.
|
||||
|
||||
To install using pip::
|
||||
|
||||
$ pip install queuelib
|
||||
|
||||
To install using easy_install::
|
||||
|
||||
$ easy_install queuelib
|
||||
|
||||
If you have downloaded a source tarball you can install it by running the
|
||||
following (as root)::
|
||||
|
||||
# python setup.py install
|
||||
|
||||
FIFO/LIFO disk queues
|
||||
=====================
|
||||
|
||||
Queuelib provides FIFO and LIFO queue implementations.
|
||||
|
||||
Here is an example usage of the FIFO queue::
|
||||
|
||||
>>> from queuelib import FifoDiskQueue
|
||||
>>> q = FifoDiskQueue("queuefile")
|
||||
>>> q.push(b'a')
|
||||
>>> q.push(b'b')
|
||||
>>> q.push(b'c')
|
||||
>>> q.pop()
|
||||
b'a'
|
||||
>>> q.close()
|
||||
>>> q = FifoDiskQueue("queuefile")
|
||||
>>> q.pop()
|
||||
b'b'
|
||||
>>> q.pop()
|
||||
b'c'
|
||||
>>> q.pop()
|
||||
>>>
|
||||
|
||||
The LIFO queue is identical (API-wise), but importing ``LifoDiskQueue``
|
||||
instead.
|
||||
|
||||
PriorityQueue
|
||||
=============
|
||||
|
||||
A discrete-priority queue implemented by combining multiple FIFO/LIFO queues
|
||||
(one per priority).
|
||||
|
||||
First, select the type of queue to be used per priority (FIFO or LIFO)::
|
||||
|
||||
>>> from queuelib import FifoDiskQueue
|
||||
>>> qfactory = lambda priority: FifoDiskQueue('queue-dir-%s' % priority)
|
||||
|
||||
Then instantiate the Priority Queue with it::
|
||||
|
||||
>>> from queuelib import PriorityQueue
|
||||
>>> pq = PriorityQueue(qfactory)
|
||||
|
||||
And use it::
|
||||
|
||||
>>> pq.push(b'a', 3)
|
||||
>>> pq.push(b'b', 1)
|
||||
>>> pq.push(b'c', 2)
|
||||
>>> pq.push(b'd', 2)
|
||||
>>> pq.pop()
|
||||
b'b'
|
||||
>>> pq.pop()
|
||||
b'c'
|
||||
>>> pq.pop()
|
||||
b'd'
|
||||
>>> pq.pop()
|
||||
b'a'
|
||||
|
||||
RoundRobinQueue
|
||||
===============
|
||||
|
||||
Has nearly the same interface and implementation as a Priority Queue except
|
||||
that each element must be pushed with a (mandatory) key. Popping from the
|
||||
queue cycles through the keys "round robin".
|
||||
|
||||
Instantiate the Round Robin Queue similarly to the Priority Queue::
|
||||
|
||||
>>> from queuelib import RoundRobinQueue
|
||||
>>> rr = RoundRobinQueue(qfactory)
|
||||
|
||||
And use it::
|
||||
|
||||
>>> rr.push(b'a', '1')
|
||||
>>> rr.push(b'b', '1')
|
||||
>>> rr.push(b'c', '2')
|
||||
>>> rr.push(b'd', '2')
|
||||
>>> rr.pop()
|
||||
b'a'
|
||||
>>> rr.pop()
|
||||
b'c'
|
||||
>>> rr.pop()
|
||||
b'b'
|
||||
>>> rr.pop()
|
||||
b'd'
|
||||
|
||||
|
||||
Mailing list
|
||||
============
|
||||
|
||||
Use the `scrapy-users`_ mailing list for questions about Queuelib.
|
||||
|
||||
Bug tracker
|
||||
===========
|
||||
|
||||
If you have any suggestions, bug reports or annoyances please report them to
|
||||
our issue tracker at: http://github.com/scrapy/queuelib/issues/
|
||||
|
||||
Contributing
|
||||
============
|
||||
|
||||
Development of Queuelib happens at GitHub: http://github.com/scrapy/queuelib
|
||||
|
||||
You are highly encouraged to participate in the development. If you don't like
|
||||
GitHub (for some reason) you're welcome to send regular patches.
|
||||
|
||||
All changes require tests to be merged.
|
||||
|
||||
Tests
|
||||
=====
|
||||
|
||||
Tests are located in `queuelib/tests` directory. They can be run using
|
||||
`nosetests`_ with the following command::
|
||||
|
||||
nosetests
|
||||
|
||||
The output should be something like the following::
|
||||
|
||||
$ nosetests
|
||||
.............................................................................
|
||||
----------------------------------------------------------------------
|
||||
Ran 77 tests in 0.145s
|
||||
|
||||
OK
|
||||
|
||||
License
|
||||
=======
|
||||
|
||||
This software is licensed under the BSD License. See the LICENSE file in the
|
||||
top distribution directory for the full license text.
|
||||
|
||||
Versioning
|
||||
==========
|
||||
|
||||
This software follows `Semantic Versioning`_
|
||||
|
||||
.. _Scrapy framework: http://scrapy.org
|
||||
.. _scrapy-users: http://groups.google.com/group/scrapy-users
|
||||
.. _Semantic Versioning: http://semver.org/
|
||||
.. _nosetests: https://nose.readthedocs.org/en/latest/
|
||||
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue