Tag: CPAN

Categories
Perl

Some Github and Perlanet news

This is just a quick update, to let you know that the code from the Perlanet Simple walkthrough is now on Github, here! There are a few changes I’ve made, though mainly just re-organise the folder to make more sense. So now the actual script is in the ‘bin’ folder, the template and config files are in ‘etc’, and the output folder is the same as before, in ‘www’. This also means that the script and config have been changed slightly, to reflect these folder changes.

The original writer of the module, Dave Cross, has also asked if he can use this tutorial in the documentation, and so eventually you will see some of this somewhere in the Perlanet module. For now though, I will be working on the next stage of this tutorial!

If anyone has anything they would like to see done with Perlanet, feel free to ask and I will see if i can accomodate it.

Categories
Perl Shadowcat

Perlanet not-quite-so Simple

So in my current project at Shadowcat, I am using a CPAN module called Perlanet, as written by Dave Cross. This module/program is mainly for aggregating web feeds (Rss or Atom feeds) and creating a new feed and web page from them.

Now, with the stuff I’m doing for Shadowcat, I have been refactoring a lot of my code from the IronMan codebase, and customising it to do what I need. However, this did lead to a lot of ‘How the hell did that actually run?’ moments, as I didn’t fully understand the underlying modules. So to fix this, I am going to do some messing with Perlanet of my own, and hopefully have a useful walkthrough at the end of it.

Either that, or this will be an entertaining read of how to bash your head against a desk… so. To Coding!

Stage one: Throw out the current docs

Now, as with most CPAN modules, there is some documentation shipped with each piece of code. And, as with all documentation, sometimes its not massively useful. Now, this might just be me being thick, but when coming to use Perlanet::Simple, I had several problems even getting off the starting block. In the docs for Perlanet::Simple, it listed that to use it, all you need to do is:

my $perlanet = Perlanet::Simple->new_with_config('perlanet.yaml');
$perlanet->run;

Now, of course you will need to include the module with ‘use Perlanet::Simple’, and need a file called ‘perlanet.yaml’ in the same directory, however apart from that… what? Well, turns out you need a few other things to make this not throw any errors…

use Perlanet::Simple;

my $perlanet = Perlanet::Simple->new_with_config(
configfile => 'perlanet.yaml'
);

$perlanet->run;

(note: I have left out ‘use strict;’ and ‘use warnings;’ of all code snippets to save space, but they are implied in all snippets… plus you’d be mad not to use them anyway).

The main (and only) change is to make the call to ‘new_with_config’ a hash, by adding “configfile => ‘perlanet.yaml'” instead of just the filename (note – ‘new_with_config’ is not defined in Perlanet code, it is actually defied in MooseX::ConfigFromFile, and expects a hash to be passed to it).

Phew. well, as that now works, onto the rest!

Stage two: Configfile? What Configfile?!

The next stage to getting this fully working, is to create the config files that Perlanet will understand, which also means knowing what you want to aggregate. There are some demo files given in the examples folder on CPAN, however it’s probably easier to see whats going on when you create your own. So, to do this you need to have or know the following:

  • A title for your page
  • A subtitle or description of your page
  • The URL for your website
  • your name (you do know this, right?) and an e-mail address
  • The number of entries you want to have on the page
  • Where you want the generated page stored and the template to use for it
  • Where you want the generated feed stored and what format to use
  • An URL, title, and website for each feed you want to aggregate

You will also need a template file to use, which is in the TemplateToolkit format. I won’t go into how to create one of those, just use the one below…

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>[% feed.title %]</title>
</head>
<body>
<h1>[% feed.title %]</h1>
<p>[% feed.description %]</p>
[% FOREACH entry IN feed.entries %]
<h2><a href="[% entry.link | url | html %]">[% entry.title | html %]</h2></a>
[% entry.content.body %]
[% IF entry.author OR entry.issued %]
<p>Published[% IF entry.author %] by [% entry.author | html; END %]
[% IF entry.issued %] on [% entry.issued | html; END %]</p>
[% END %]
[% END %]
<hr />
<address>[% feed.author | html %] / [% feed.modified | html %]</address>
</body>
</html>

Now for this example, I will make something that aggregates 3 blogs – Makezine, Arduino, and Adafruit. These are mainly because I love making things, and these are 3 very popular blogs for Makers in general, but you could use any other blogs you want! All you need is the RSS or Atom feed link.

First thing, create a file called ‘perlanet.yaml’ in the same directory as your script. then add the following lines, populating it with your own bits as required:

title: Maker Planet
description: Make all the things!
url: http://tbsliver.wordpress.com
author:
name: Tom Bloor
email: not.so.stupid@toputthis.here

This info is just the basic bits for the actual page you’re creating. The next few lines are for defining your input and output files:

# previous bits go here
entries: 20
page:
file: www/index.html
template: index.tt
feed:
file: www/atom.xml
format: Atom

In this part, you have defined the number of entries you want in your feed (here its 20, Perlanet actually defaults to a pretty sane 30, though ofcourse you can have this as high or as low as you like), the output file you want the html in (here it will be put in a folder called ‘www’) and the template to use for this file (see Template part earlier). The last part is completely optional, but will output an Atom feed file that you can then subscribe to! (You can also make this RSS by changing the format to RSS).

After these, you then add the feeds that you want!

# previous bits go here
feeds:
- url: http://blog.makezine.com/feed/
- url: http://arduino.cc/blog/feed/
title: Arduino Blog
web: http://arduino.cc/
- url: http://www.adafruit.com/blog/feed/
title: Adafruit Industries

Here it shows the 3 feeds i chose earlier. It also shows the extra information you can add to each, though the title and web parts are completely optional – I think those are only used if you have them in your Template. (Or if you customise Perlanet, but that I’l go into later…).

With all those bits, you can now run your Perlanet, and it should just work! (as always, mileage may vary…)

Stage three: Bug Stomping

Phew! you made it this far. Now then, if you have managed to run your script, and it didnt throw any errors, and the web page that came out is respectable (Ok it may look pretty crap if you used the template I provided, however it IS very basic anyway…), then you may not need the few things I’ve learnt while doing this. However, if you have a few bugs to work out, then here are the few things I found worked.

  • Double check your perlanet.yaml file. Seriously, as you’ve created it by hand, there is most likely an error, a typo, a missed variable, something. Also check the whitespace – look at the example from CPAN (here) to see how it should be spaced.
  • If the web page looks especially bad, see if it isn’t just some stray style bits from the feed you have aggregated – they sometimes have spacing things in there that will screw with the layout. Will go into more detail later how to filter that out, for now… sorry.
  • Lastly, it may actually be an issue with the code I’ve posted. If nothing you try fixes the issue, or you find I have a mistake in my code, feel free to comment and I’l get back to you when I can, or fix the issue in my code. Other options, are to go to the many other Perl resources that are around, such as IRC, or the many other websites (too many to list…) that may be able to help.

Well that about sums it up! Next time, I will be going into more detail of changing the workings of Perlanet to do some more useful things. I will also probably go through some changes to the file and folder structure of this project to be a bit more sane, though for now it isn’t a massive project so probably doesn’t matter as much. enjoy!

Edits:

Spelling mistake spotted by castaway (your -> you’re).
removed the ‘use Perlanet::Trait::YAMLConfig;’ as seems to not need it… random bug I had thats disappeared since.
Fixing highlighting items