Download: rawdog-2.23.tar.gz (68K, released 2018-12-26)
Signature (about); SHA1 171520c909fae81ac8ec0e063e7ca36593aae372

Available through Git (Atom feed of changes):
git clone http://offog.org/git/rawdog.git

About rawdog

rawdog is an RSS Aggregator Without Delusions Of Grandeur. It is a "river of news"-style aggregator: it uses feedparser to download feeds in RSS, Atom and a variety of other formats, and (by default) produces static HTML pages containing the newest articles in date order. For example, rawdog's default configuration produces output that looks like this:

Screenshot of rawdog output

rawdog is designed to be invoked periodically by cron (or a similar task-scheduling mechanism). It stores data in flat files, so no database is required. It can use threads to download multiple feeds in parallel, and uses ETags, Last-Modified, and gzip compression to minimise network bandwidth usage.

Written in Python, rawdog is highly customisable and extendable. Its output is generated using templates, so it's easy to integrate the output of rawdog into your own web pages. You can extend rawdog's functionality using plugin modules written in Python — for example, you can provide output in different formats, or filter or modify incoming data from feeds.

Full documentation is included in the source package to get you started with rawdog. If you're using a binary package that doesn't include the documentation, you can view the latest README, config and style.css files here.

Dependencies

rawdog requires Python 2.7 or later, and feedparser 5.1.2 or later. (rawdog won't work with Python 3.x; Python 3 is a different language from Python 2.)

I'd also strongly recommend installing PyTidyLib, which rawdog can use to generate cleaner output HTML.

rawdog is designed to run on POSIX-compliant free operating systems such as Linux and FreeBSD, and requires Unix-like filesystem semantics. rawdog is not supported on proprietary operating systems.

Mailing list

There is a mailing list for rawdog users and developers, upon which new versions of rawdog are announced and discussed.

Packaged versions of rawdog

rawdog packages are available for some systems. These are usually more convenient for end-users to install than the source tarball above, but they might not contain the very latest version of rawdog.

For Debian, the rawdog package was originally developed by Decklin Foster, and is now maintained by me. (The debian directory for the rawdog package is available in git.)

For Fedora, Andrew Grover maintains a rawdog package.

For FreeBSD, you can install the news/rawdog port, maintained by Tim Bishop.

For Gentoo, rawdog is available as net-news/rawdog.

For OpenBSD, Xavier Santolaria maintains the www/rawdog port.

Plugins

Available through Git (Atom feed of changes):
git clone http://offog.org/git/rawdog-plugins.git

Here are the plugins that are currently available; if you've written rawdog plugins that you'd like to be listed here, please let me know. If you can't see a plugin to do what you want, you can probably write it yourself without too much effort. See the PLUGINS file in the source package for more details of the plugin API, or look at the existing plugins provided. If you're not sure where to start, please contact me.

To install a plugin, make sure that you have plugindirs plugins in your config file, and drop the plugin into your ~/.rawdog/plugins directory.

Name Maintainer Purpose
archive Adam Sampson Write incoming articles in Atom format to a local archive (needs my atomwriter module)
article-filter Adam Sampson Filter articles on a per-feed basis using regular expressions
article-stats Adam Sampson Print counts of articles added, updated, expired and stored when rawdog exits
author-no-link Stephan Manske Provide author with HTML removed for templates
backwards Adam Sampson Example plugin: reverse the article sort order
dated-output Adam Sampson Split output into dated files
detwit Adam Sampson Remove prefixes from Twitter messages
digest-auth Adam Sampson Add HTTP digest authentication support
download-articles Adam Sampson Download copies of articles automatically
enclosure Virgil Bucoci Display links to article enclosures
feed-execute Adam Sampson Execute commands before or after fetching feeds
feedgrep Steve Atwell Filter articles by regular expression
feedgroup Zephaniah E. Hull Index articles by user-defined groups
feedwise Virgil Bucoci Group articles by feed rather than by date
imgstrip Virgil Bucoci Replace images with links
inline_link Brian Jaress Provide data: links to long article descriptions
links TheCrypto Add static links to rawdog's output
ljkludge Adam Sampson Work around a bug in LiveJournal digest authentication
paged-output Adam Sampson Split output into several smaller files (needs rawdog 2.5rc1 or later)
printnew Ted T Send new articles by email
rss Jonathan Riddell Export RSS, FOAF and OPML automatically for all feeds
select-feeds Adam Sampson Select feeds to include in the output
sidebarfeedwise Gutemberg A. Vieira Feed-grouped display with a sidebar
since-last Adam Sampson When writing, only include articles that haven't previously been written
slashdot Virgil Bucoci Display Slashdot's department lines in articles
status-log Adam Sampson Keep and display a log of errors when fetching feeds
tagcat Decklin Foster Organise feeds using user-defined tags
truncate Adam Sampson Truncate and/or strip HTML markup from articles
vellum-templates Adam Sampson Replace rawdog's templating system with the one from Vellum
xml_archiver BAM Write rawdog's article database out as XML for external processing

Styles

The appearance of rawdog's output can be customised using CSS stylesheets and HTML templates. rawdog includes a default style (shown in the screenshot above), but here are some more that rawdog users have contributed. If you've come up with a new stylesheet or template that others might like, please let me know.

Name Author
dec23 Adam Sampson
shym Samuel Hym

Related work

Wari has written an xscreensaver fortune program that retrieves random articles from rawdog 1.x.

Nimrod's Dogwalker is a modified version of rawdog 1.x that supports FTP uploading and RTL languages like Hebrew.

Steve Pomeroy's rawdog to OPML exporter converts rawdog config files to OPML, with categories.

Conversely, Tero Karvinen's opml-export-feeds script can turn an OPML list into a rawdog config fragment.

Austin Bingham's rded provides a web-based interface for editing your rawdog config.

Brian M. Clapper's curn is very much like rawdog, but written in Java. Neither of us were aware of the other's project!

Planet KDE uses rawdog, with some customisations by Jonathan Riddell; the code and configuration are in KDE's repository.

If you've done something cool related to rawdog, let me know so I can add a link here to your project.