Categories
Perl Shadowcat

Perlanet not-quite-so Simple

So in my current project at Shadowcat, I am using a CPAN module called Perlanet, as written by Dave Cross. This module/program is mainly for aggregating web feeds (Rss or Atom feeds) and creating a new feed and web page from them.

Now, with the stuff I’m doing for Shadowcat, I have been refactoring a lot of my code from the IronMan codebase, and customising it to do what I need. However, this did lead to a lot of ‘How the hell did that actually run?’ moments, as I didn’t fully understand the underlying modules. So to fix this, I am going to do some messing with Perlanet of my own, and hopefully have a useful walkthrough at the end of it.

Either that, or this will be an entertaining read of how to bash your head against a desk… so. To Coding!

Stage one: Throw out the current docs

Now, as with most CPAN modules, there is some documentation shipped with each piece of code. And, as with all documentation, sometimes its not massively useful. Now, this might just be me being thick, but when coming to use Perlanet::Simple, I had several problems even getting off the starting block. In the docs for Perlanet::Simple, it listed that to use it, all you need to do is:

my $perlanet = Perlanet::Simple->new_with_config('perlanet.yaml');
$perlanet->run;

Now, of course you will need to include the module with ‘use Perlanet::Simple’, and need a file called ‘perlanet.yaml’ in the same directory, however apart from that… what? Well, turns out you need a few other things to make this not throw any errors…

use Perlanet::Simple;

my $perlanet = Perlanet::Simple->new_with_config(
configfile => 'perlanet.yaml'
);

$perlanet->run;

(note: I have left out ‘use strict;’ and ‘use warnings;’ of all code snippets to save space, but they are implied in all snippets… plus you’d be mad not to use them anyway).

The main (and only) change is to make the call to ‘new_with_config’ a hash, by adding “configfile => ‘perlanet.yaml'” instead of just the filename (note – ‘new_with_config’ is not defined in Perlanet code, it is actually defied in MooseX::ConfigFromFile, and expects a hash to be passed to it).

Phew. well, as that now works, onto the rest!

Stage two: Configfile? What Configfile?!

The next stage to getting this fully working, is to create the config files that Perlanet will understand, which also means knowing what you want to aggregate. There are some demo files given in the examples folder on CPAN, however it’s probably easier to see whats going on when you create your own. So, to do this you need to have or know the following:

  • A title for your page
  • A subtitle or description of your page
  • The URL for your website
  • your name (you do know this, right?) and an e-mail address
  • The number of entries you want to have on the page
  • Where you want the generated page stored and the template to use for it
  • Where you want the generated feed stored and what format to use
  • An URL, title, and website for each feed you want to aggregate

You will also need a template file to use, which is in the TemplateToolkit format. I won’t go into how to create one of those, just use the one below…

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>[% feed.title %]</title>
</head>
<body>
<h1>[% feed.title %]</h1>
<p>[% feed.description %]</p>
[% FOREACH entry IN feed.entries %]
<h2><a href="[% entry.link | url | html %]">[% entry.title | html %]</h2></a>
[% entry.content.body %]
[% IF entry.author OR entry.issued %]
<p>Published[% IF entry.author %] by [% entry.author | html; END %]
[% IF entry.issued %] on [% entry.issued | html; END %]</p>
[% END %]
[% END %]
<hr />
<address>[% feed.author | html %] / [% feed.modified | html %]</address>
</body>
</html>

Now for this example, I will make something that aggregates 3 blogs – Makezine, Arduino, and Adafruit. These are mainly because I love making things, and these are 3 very popular blogs for Makers in general, but you could use any other blogs you want! All you need is the RSS or Atom feed link.

First thing, create a file called ‘perlanet.yaml’ in the same directory as your script. then add the following lines, populating it with your own bits as required:

title: Maker Planet
description: Make all the things!
url: http://tbsliver.wordpress.com
author:
name: Tom Bloor
email: not.so.stupid@toputthis.here

This info is just the basic bits for the actual page you’re creating. The next few lines are for defining your input and output files:

# previous bits go here
entries: 20
page:
file: www/index.html
template: index.tt
feed:
file: www/atom.xml
format: Atom

In this part, you have defined the number of entries you want in your feed (here its 20, Perlanet actually defaults to a pretty sane 30, though ofcourse you can have this as high or as low as you like), the output file you want the html in (here it will be put in a folder called ‘www’) and the template to use for this file (see Template part earlier). The last part is completely optional, but will output an Atom feed file that you can then subscribe to! (You can also make this RSS by changing the format to RSS).

After these, you then add the feeds that you want!

# previous bits go here
feeds:
- url: http://blog.makezine.com/feed/
- url: http://arduino.cc/blog/feed/
title: Arduino Blog
web: http://arduino.cc/
- url: http://www.adafruit.com/blog/feed/
title: Adafruit Industries

Here it shows the 3 feeds i chose earlier. It also shows the extra information you can add to each, though the title and web parts are completely optional – I think those are only used if you have them in your Template. (Or if you customise Perlanet, but that I’l go into later…).

With all those bits, you can now run your Perlanet, and it should just work! (as always, mileage may vary…)

Stage three: Bug Stomping

Phew! you made it this far. Now then, if you have managed to run your script, and it didnt throw any errors, and the web page that came out is respectable (Ok it may look pretty crap if you used the template I provided, however it IS very basic anyway…), then you may not need the few things I’ve learnt while doing this. However, if you have a few bugs to work out, then here are the few things I found worked.

  • Double check your perlanet.yaml file. Seriously, as you’ve created it by hand, there is most likely an error, a typo, a missed variable, something. Also check the whitespace – look at the example from CPAN (here) to see how it should be spaced.
  • If the web page looks especially bad, see if it isn’t just some stray style bits from the feed you have aggregated – they sometimes have spacing things in there that will screw with the layout. Will go into more detail later how to filter that out, for now… sorry.
  • Lastly, it may actually be an issue with the code I’ve posted. If nothing you try fixes the issue, or you find I have a mistake in my code, feel free to comment and I’l get back to you when I can, or fix the issue in my code. Other options, are to go to the many other Perl resources that are around, such as IRC, or the many other websites (too many to list…) that may be able to help.

Well that about sums it up! Next time, I will be going into more detail of changing the workings of Perlanet to do some more useful things. I will also probably go through some changes to the file and folder structure of this project to be a bit more sane, though for now it isn’t a massive project so probably doesn’t matter as much. enjoy!

Edits:

Spelling mistake spotted by castaway (your -> you’re).
removed the ‘use Perlanet::Trait::YAMLConfig;’ as seems to not need it… random bug I had thats disappeared since.
Fixing highlighting items

Categories
linux Programming Shadowcat

The Cat of Shadow, and Multiplexed Terminals

So for those of you who don’t know (which will probably be most of you…), I have started interning at Shadowcat Systems, where their philosophy (for new staff atleast) is to push you off a cliff, and see how long it takes for you to learn to fly. Currently im still falling, though slightly slower than before, which brings me to the main subject of this post: Tmux.

So, way back (ok so more like mid 2007/8) when I started messing in Linux for desktop and server based stuff, I had the need for keeping terminal’s open even when I was not connected to the server, and for having several open at one time. Now anyone who has used Linux for such purposes have probably used something called Screen – an exceedingly powerful window manager for the command line, though with a very steep learning curve, and (as i found ‘back in the day’) quite a difficult to understand config and use.

Fast forward to a few months ago, when I was first sorting out things with Shadowcat, and I realised that a lot of the time there I would be spending at the command line, and would have need for something quite similar to Screen. Remembering how little I liked said program, I had a look around, and happened across Tmux. Now dont ask me how I came across it, I really cannot remember the details, but all I know is that it makes sense. Ok, as with all command line interfaces and the like, there is always a learning curve, and I dont know how much of this I remembered from using Screen, but it clicked in and worked.

One of the major things I liked, was that there was a permanent statusbar, just like the taskbar seen in Windows/ Linux/ Mac/ insert OS here. I know that screen has the same option, but at the time I didnt know that, and having looked at some of the documentation for how to make that work nicely, I’m quite sure I would have just ran away screaming. Another function I liked was the ability to split windows into ‘panes’, and swapping between them quickly. Again, screen has this functionality (I think, somewhere….), but as before, being able to just pick it up and run with it is always a good sign.

The other major thing that I really like about it now, is how easy it is to customise it. I spent quite a while looking at tmux-powerline, which is a plugin of sorts that is designed to change the look of the statusbar, and update it with loads of quite useful functions. However, I really did not need all that extra stuff, and to be honest was quite confused about the ‘patched fonts’ – which relied on another related project’s documentation and files to work (the fact that the instructions linked to on the other projects documentation no longer exists, and when eventually found on ANOTHER related project, didn’t work either, did not bode well…). And anyway, these fonts were just for a few bits of visual flair… nice, but I felt was unnecessary, and was not going to work in my use-case either. The one major thing I liked about it, was the colour and the rough look of the bar, which I unceremoniously nabbed and then played with until I had what I am currently using.

tmux-current
My current Tmux scheme

As you can see, if you have taken a look at the powerline thingy linked earlier, the colours are quite similar. The one major difference though, is that this doesn’t need any shell scripts to run, and all is in the .tmux.conf file, which I can copy to any machine im working on which has Tmux, and have an environment that I am happy to use all day long. My Tmux config is below, and if anyone would like me to go through it, or have any questions about it, feel free to ask.

#-----------------------------------------------------------
# Tmux Config
#
# Created by Tom &quot;TBSliver&quot; Bloor
#
# Provided as-is, do what you want with it.
# Usual not-my-fault disclaimer if something goes wrong
# after using this!
#-----------------------------------------------------------

# Set Ctrl - a as the standard prefix, globally
set -g prefix C-a

# Unbind Ctrl - b as a shortcut so can use it for something else
unbind C-b

# Bind Ctrl - a as the send-prefix command to stop confusion
# with vim or when using tmux in tmux
bind C-a send-prefix

# set the dafault delay to make tmux more responsive
set -sg escape-time 1

# bind r to reload this conf file
bind r source-file ~/.tmux.conf \; display &quot;Reloaded!&quot;

# set the terminal to use 256 colors
set -g default-terminal &quot;screen-256color&quot;

# set what the current active window status bar should look like
set-window-option -g window-status-current-format &quot;#[fg=colour255, bg=colour27] #I : #W &quot;

# set what the standard status format should be
set-window-option -g window-status-format &quot; #I : #W &quot;

## Set the text (fg) and background
set -g status-fg colour136
set -g status-bg colour235

# set the window status colors and attribute
setw -g window-status-fg default
setw -g window-status-bg default
setw -g window-status-attr dim

# set the pane borders and colors
set -g pane-border-fg colour136
set -g pane-border-bg colour235
set -g pane-active-border-fg colour235
set -g pane-active-border-bg colour136

# set the alert and tmux command line colors
set -g message-fg colour136
set -g message-bg colour235
set -g message-attr bright

# set the window list to centre
set -g status-justify centre

#set utf8 encoding
set -g status-utf8 on

# set the left side of the status bar to show the current active
# session, window and pane
set -g status-left &quot;#[fg=colour234, bg=colour148] #S:#I.#P #[fg=colour0, bg=colour33] #H &quot;

# set the right side of the status bar to show the current date and time
set -g status-right &quot;#[fg=colour136, bg=color235] %a %d-%m-%Y | %H:%M &quot;

# set tmux to monitor for activity in a window, and let the
# window list show activity in a window
setw -g monitor-activity on
set -g visual-activity on