External Hard Disk setup

I recently purchased a Seagate 250 GB external USB hard disk to use as a backup medium for my home network. This will be connected to my main Linux box (currently running Kubuntu Breezy) and important personal data and system files will be rsync‘d to it on a regular basis.

Herewith some notes on configuration, somewhat distro-specific although not uselessly so. (At least I hope not, as I plan to rebuild my home server fairly soon and may well switch. I’ve been experimenting with Kubuntu after years as a Slackware user and have not settled on a final decision just yet.) On that note: please bear in mind that this post is not intended as a definitive HOWTO but should be considered as my notes on what I did to get this working for me under a particular set of circumstances, with far less time to research everything involved than I’d really have liked. In other words: Your Milage May Vary.

The disk comes preformatted with the somewhat limited FAT32 filesystem. Given that all files being backed-up will be coming from systems running one flavour or another of Linux and that the disk will probably only need to be accessed directly from Linux systems, I’m going to reformat with ext3 using the following steps:

  1. Plug in the disk. Kubuntu detects it and pops open a window asking what to do. I select “Open in a new Window” initially which mounts the disk at /media/sda1 and opens a Konqueror window at system:/media/sda1 to view the volume.
  2. Looks fine. I close Konqueror, open a Konsole window and type sudo fdisk /dev/sda at the prompt to check the partiton table of the disk (type p at the fdisk prompt), which contains no surprises as the disk is one large FAT32 volume. Type q to quit fdisk.
  3. Unmount the disk manually : sudo umount /dev/sda1. This needs to be done in order to format the disk.
  4. sudo mkfs.ext3 /dev/sda1. Wait a couple of minutes for the format to complete.
  5. Check it mounts ok: sudo mount -t ext3 /dev/sda1 /media/sda1. Great.
  6. Check it still automounts properly on power-on by unmounting manually, turning the disk off, then turning it back on again. All fine.
  7. In the default state, the disk is mounted on a dynamically-created directory under /media named for whichever device node it is assigned when it’s plugged in. This means that the mount point may change, making the task of writing scripts to automate the backup procedure more complicated. The following steps go a long way to ensuring that the disk is always mounted in the same place:
    1. Added the following entry to /etc/udev/rules.d/hal.rules (more information on writing udev rules):
      BUS=="usb", SYSFS{product}=="Seagate External Drive", \
      KERNEL=="sd?1", NAME=="%k", \
      SYMLINK=="seagate", GROUP=="hal"
    2. sudo mkdir /media/seagate
    3. Added the following line to /etc/fstab:
      /dev/seagate    /media/seagate  ext3    noauto,rw,user 0 0

    You need to restart your hald for this to take effect. I ended up rebooting, but I couldn’t see why something like /etc/init.d/dbus restart wouldn’t do the trick.

    It’s worth noting that although this does mount the disk under /media/seagate, it still shows up in Konqueror at system:/media/sda1. Not sure why at time of writing – this feels like a bug.

    Also note that I used e2label to the give the device a label (“SEAGATE”). I was initally looking at using this to mount the volume to the desired mount point with a line in fstab starting LABEL=SEAGATE, but the udev/hal fix turned out to be the simplest option(!)

  8. mkdir some top level folders: music, photos and home. Then do an initial series of rsync runs, starting with a simple rsync -av /media/photos/ /media/sda1/photos/. Everything sysncs fine. Now I’ll be able to backup quickly as and when necessary, and write some scripts to automate everything … eventually.

More on securing SSH

Just a brief post to flag that I’ve re-written my earlier post on securing SSH servers to include quite a lot more detail. I’d be very interested in any feedback and suggestions on this, particularly stuff I’ve missed out or any mistakes that may have slipped in, so please get in touch if you’ve any comments (I really must implement a commenting system here at some point).

Hoary to Breezy

I just upgraded the Kubuntu Hoary Hedgehog partition on my iBook to Breezy Badger. I left it a while to allow any obvious problems with the new release to be ironed out. What a breeze it proved to be:

$ sudo sed s/hoary/breezy/g -i /etc/apt/sources.list

$ sudo apt-get update

$ sudo apt-get dist-upgrade

Then it took about 36 minutes to download the 500-odd Megs of required files over my 2 Mbps line and about another 20 or 30 minutes to perform the upgrade. During this time, it asked me three questions: permission to stop some services, what default language to use, and whether to replace the global Xsessions file.

Everything looked fine. Everything still seemed to be working. A reboot resulted in a nice new login screen and no odd behaviour. Logging-in took me to my familiar desktop. Touch wood, but it all seems to have Just Worked. Very impressive.

The Bristol Wireless LTSP suite

Bristol Wireless are a collective who run, among other things, a community-access wireless network in the Easton area of the city. Their LTSP suite is one of those other things.

I’ve been working with some of the Bristol Wireless crew at The St Werburghs Centre over the last few months, but it wasn’t until FAVE last Saturday that I got a chance to see the LTSP suite up and running, and I was so impressed I thought I’d write about it.

LTSP stands for the “Linux Terminal Server Project”. Simply put, a terminal server is a machine that runs applications on behalf of client machines – the terminals. This means you can use old or low-spec hardware to build clients and connect them to a more powerful machine which will actually do most of the work, enabling users to access applications on the clients that they’d be too slow or flakey to run without the server’s power.

[LTSP users at FAVE]

The Bristol Wireless LTSP suite is just such a patchwork of near-obsolete and flashy new hardware: a pile of ancient laptops donated by the local police force are connected to a powerful new Acer laptop boasting gigabytes of RAM using a bundle of Cat-5 and a gigabit ethernet switch. The Acer takes care of the bulk of the processing and the gigabit ethernet ensures that the connections are as fast as possible. Combine it with the satellite uplink from their collaborators Psand, and they’ve got a mobile IT facility that fits into the boot of a small car!

[The Terminal Server]

And it works like a charm. The clients are responsive and boast a wide range of applications and the whole system has proved a success with users: so far this summer it’s been taken to the Home Education Festival and the Big Green Gathering where it was a hit with the kids as well as the FAVE event where it proved popular among the geeks (like me, for example). As well as being a flexible community resource, the project is a great showcase for what can be achieved with some old hardware, Free software and a bit of ingenuity, and it provides a slick and impressive user experience for people coming to a Free software based system for the first time.

[More LTSP users at FAVE]

Given the fact that Bristol Wireless come across as a relaxed group who run their projects on an ad-hoc voluntary basis, this shows what they can put together when they get down to business. They are using LTSP to build IT suites elsewhere in the city and are expanding their operations to offer other services such as internet connectivity as well as access to the local wireless network (my employer, the St Werburghs Centre, is one of their new customers), so let’s hope their future projects continue to be as impressive.

(For more photos, see here.)

Brute force ssh password attacks

Since opening my SSH server at home to the internet yesterday – less than 24 hours ago – there have been 2883 failed attempts to log in from two ip addresses: 209.183.190.11 and 59.120.13.130. That makes me nervous, even though I’m reasonably confident that it’s secure due to password authentication being turned off and the following two handy iptables rules in force:

iptables -A INPUT -p tcp -m state --state NEW --dport 22 \ 
 -m recent --update --seconds 15 -j DROP 

iptables -A INPUT -p tcp -m state --state NEW --dport 22 \
 -m recent --set -j ACCEPT

I got this idea from a post to alt.os.linux.slackware. These rules use the recent extension to iptables to track attempted SSH connections and drop any that come from the same IP within 15 seconds. If you miss-type your passphrase, you just have to remember to hang back for a few seconds before retrying to avoid getting caught out. You could jump to a custom chain that logged such connections before dropping them if you felt the need to monitor the effectiveness of the rule – although the scripts usually just stop trying after a few connection attempts are dropped.

Worm related spamming

I got an email this morning from “Daphne Jacobsen”, a marketroid at a CD/DVD company that shall not be getting any plugs here. She claimed that someone from my company had mailed them requesting prices last week but that their servers had become infected with one of the many worms on the lose recently and that the message had been lost (but obviously not completely lost, otherwise where did she get the email address?) Her message ended:

In case you need more information, our company web site is [DELETED] where you can see we are a complete “one stop shop” for DVD, CDROM, printing, packaging, and fulfillment services.

If you need, please call me TOLL FREE at [DELETED].

Obviously spam. The mail was sent to an email address that I’ve never used at a domain I’ve only ever used for personal purposes. Interesting, though. The mail was obviously carefully written to sound genuine and unique. Hand-wringing over the problems caused by the “worm” ties it in nicely with current events on the internet and might make a receipient feel sympathetic to the sender. At first glance, not your usual spam – possibly different enough to not only escape spamtraps (it slipped past two to get to me) but to snare a few more unwary punters than usual. I’ve never received anything quite so carefully crafted before (if you exclude some of the better phishing emails).

Basic version control with wikieditish

At the St Werburghs Community Centre, we use Blosxom to power our website. As the other staff needed to be able to update and add content quickly and with the minimum of fuss, I combined the wikieditish plugin with TinyMCE and a couple of other scripts to create a simple rich-text web interface to the site.

During testing and the initial setup period, it became apparent that having the ability to roll an entry back to its original state after a dodgy edit would be a useful feature, so I added a few lines of code to wikieditish that supports a very simple kind of version control: every time a change is made, the plugin saves a copy of the current state of the post into a backup file. This file is over-written each time so you can only revert to the last saved version, but this allowed staff some scope for experimentation with the knowledge that they could undo any serious mistakes quickly.

I thought that I’d make this version of the plugin available here in case any other blosxom users might find it useful, so download wikieditishvc.zip, which contains the plugin and some basic flavour templates. I have not renamed the plugin itself, it is still simply named wikieditish, but I’ve added a bit of extra documentation and updated the version number to differentiate it. The version control behaviour is all optional and is off by default.

Atom 1.0 Support for Blosxom

Here is an updated atomfeed plugin that supports Atom 1.0: download atomfeed.

Changes since the last beta version:

  • Moved variable configuration to head sub
  • Added simple entity-unescaping in generation of the element
  • Adjusted the xml:base attribute to reflect $id_domain rather than $blosxom::url, and added the attribute at feed-level as well as for each entry

This plugin offers basic support for most of the specification, which should be enough for the vast majority of users. Not currently supported are the and elements. has been excluded as it seems an element more aimed at those publishing feeds aggregated from other feeds. probably could be supported, but would have been a bit of a kludge without the use of metadata – so anyone who needs this element could use the meta plugin and a custom template.

Of course, Atom is designed to be flexible and extensible and there are already people out there imagining losts of uses for the format – see this article at IBM Developer Works for some ideas (via Tim Bray). This plugin can never hope to cater for all possible uses, but it should be enough to provide the bare bones that others can build upon to meet their own needs.

The plugin comes with flavour templates and extensive documentation, including a list of all the variables it makes available for use in templates, notes on compatibility issues with other plugins and suchlike. I’ve extracted some of the major points below, but see the plugin itself for more complete notes.

  • The plugin has a large number of configurable variables and generates an even larger number of template variables. However, it is designed to be usable with only minimal configuration: set the variables $default_author and $feed_yr, drop the plugin into your $plugins_dir and off you go.
  • The plugin is intended to work with nothing but blosxom.cgi and a standard perl installation, but it will perform better if the XML::Parser and LWP modules are installed, and if you are using a plugin like entries_cache that stores the creation times of your posts.
  • If you are modifying the content of your entries using other plugins, particularly any that escape characters or add markup (like some of these), you should have them run before the atomfeed plugin. You want atomfeed to get each post in the state that it would be sent to a normal web browser.
  • Similarly, if you intend to use the config or prefs plugins to modify any of the variables generated by the plugin, they will need to run before it.
  • Podcasters: enclosures are supported. You need to link to your enclosure in the body of your weblog post, and provide the anchor tag with a rel attrubute of “enclosure”. For example: <a rel="enclosure" href="http://example.com/podcasts/august-05.mp3">Podcast for 5th August</a>. If you have the LWP module and you change the $use_full_enclosures configurable variable to “1”, the plugin will also include data on the length and content type of the enclosure (this is recommended as good practice).
  • You can include “related” and “via” links using a similar method – just ensure the anchor tags have an appropriate rel attribute for these links to be included in your feed as corresponding s.

Please post any feedback you have to the blosxom mailing list, maybe as a follow-up to this post.

Beta support for Atom 1.0 in blosxom

atomfeed released!

It’s finished! See this post for more information, docs and download.

atomfeed-beta-6

Download: atomfeed-beta-6

This is now pretty much ready for release: just a last check for suggestions and bugs. It now supports the whole spec apart from , which seems aimed at aggregators rather than bloggers, and , which could easily be implemented if desired using the meta plugin.

Changes:

  • Moved from using class to rel attributes in HTML anchors to identify enclosure/via/related links to include in the feed. Just use the appropriate rel value – there’s no need to include the “atom-” prefix any longer.
  • Support for
    . This uses a technique similar to that used by the foreshortened plugin. It is excluded from the default templates in favour of a full-text element. I’ve temporarily placed the

    inside a comment in my feed if you’re interested in how it looks.
  • and are now available as user-configurable variables.
  • Documentation is now included in the plugin – use perldoc or read directly.
  • Includes support for a stylesheet, like in the original 0.3 version. Excluded by default. You can also specify the MIME tpye of the stylesheet.

As before, comments to the mailing list please.

atomfeed-beta-5

Download atomfeed-beta-5

Changes in this version: support for rel attributes in elements, including enclosures (based on Dave Slusher‘s and Keith Irwin‘s enclosures plugin for RSS 2).

  • enlcosure: In the body of your post, link to a file you would like to appear in your atom feed as an enclosure. Make sure this link has a class attribute of “atom-enclosure”. For extra points: if you have access to the LWP perl module, set the configurable variable $use_full_enclosures to “1” and atomfeed will attempt to find the content-type and length of your enclosure. (Test: This little audio file should appear in my post as an enclosure. The code for this link looks like this: This little audio file)
  • via: To include a link from your post in your atom feed as , give it a class attribute of “atom-via”
  • related: similar to via, simply give a link a class attribute of “atom-related” for it to appear in your atom feed as an appropriate element.

Note: using the class attribute to identify these links may change in future versions.

atomfeed-beta-4

Download atomfeed-beta-4

Changes in this version (also posted to mailing list) based on suggestions from Stu MacKenzie:

  • typo corrected
  • XML::Parser is now a require rather than a use – those who don’t have this module can now use the plugin with the consequence that their entries will never be labelled as xhtml, only text or html, regardless of validity/well-formedness.
  • Regular expression for guessing text vs. mark-up is now: m!! This should hopefully catch pretty much anything that resembles mark-up.
  • The check in the start() sub for preloaded templates is now or‘d with the call to _load_templates. I’m not even sure if this check is necessary, as it doesn’t appear possible for any user-defined templates to have been loaded by this stage anyway, unless someone else knows better?

Major Revision: atomfeed-beta-3

Since posting this entry, I have made a number of changes to the plugin. It now contains support for the following additional elements:

  • and for

Also, it checks $blog_title, $blog_description and each post’s $title for markup by looking for left-side angle brackets anything matching the regular expression /<[a-zA-Z0-9]+>/. If it finds a match, it assumes that the variable contains markup and parses it as such, labelling it either html or xhtml depending on well-formedness.

You can download the new version here. The link below continues to point to the original version. Documentation is still not finished, but the code is commented. I’m particularly interested in feedback on the _parse_markup subroutine.

Original post

I’ve updated the atomfeed plugin for Blosxom to emit a basic Atom 1.0 feed. It’s in beta; you can download it here. I’m running the plugin, so if you like you can see it in action by checking my Atom feed; the feed’s been checked with feedvalidator.org‘s new Atom 1.0 support. Here are the notes I posted to the blosxom mailing list:

The feed is very basic: only the required elements along with a feed-level pointing at the feed, a based on $blog_description, a element (updated 2005-07-21, but it was always in there), and each has a as well as an element.

I have not yet updated documentation in the plugin. I’m making it available so that anyone interested can take a look and make suggestions before I start looking at adding more support for optional elements and doing some other bugfixes. When I’m happy with it I’ll re-write the docs, remove the BETA flag and post some instructions on my site. Until then, use these notes as guidance if you want to use or test the plugin:

  • $feed_yr is now a configurable variable that MUST to be set to the year you want to see in your feed level element. This should be set once and then never changed.
  • is derived from the timestamp in %blosxom::files, and (entry level) is derived from a stat()->mtime on the actual file. For these two elements to work as intended, you really should be running entries_cache or similar, otherwise they will be the same.
  • entry level ‘s are derived from %blosxom::files. Again, for maximum conformance – to guarantee the tags never change – is currently to use entries_cache or similar.
  • I decided not to worry about sniffing for plaintext entries as it seems to me that most blosxom users will have markup in their entries, so the mechanism for determining the type of has remained bascially the same (UPDATED – see below).
  • I am still assuming that $blog_title and $blog_description contain only plaintext and no mark-up whatsoever.
  • the feed-level element must appear before the s, so I’ve taken the method used in the rss10 plugin to insert it in $blosxom::output in the foot subroutine using a placeholder.

Update 2005-07-21 (1)

I’ve removed the code that conditionally placed some HTML content into a CDATA section. ALL content identfied as HTML will now be escaped.

Feedback

Please direct comments to the blosxom mailing list.

Securing sshd

I’ve decided to collate what I’ve picked-up on this topic as much of the advice I’ve found on the net is rather fragmented. Following these instructions is no substitute for reading the documentation with your SSH server and should be followed at your own risk – this is very much a brief round-up and not a detailed HOWTO. Anyway, having any services publicly available to the internet is a security risk and no amount of preparation can provide 100% certainty against compromise.

  1. Is your version of SSH up to date? Do you have a system for ensuring that it is kept up to date? None of the tips in this post will be of any use if your server has any known vulnerabilities (or any largely unknown ones for that matter, although the only way to effectively protect against these is to give up and unplug the network cable 😉 The same should apply for all the other services you make available to the internet as having a tightly locked-down SSH daemon won’t be much use if your mail and web servers are full of holes. Make sure you have a reasonable understanding of what it takes to secure your machine – see the related reading section below for some starting points.
  2. Work out how sshd is started and with what command line options. For example, are you sure you know what configuration file the server is using? The default is /etc/ssh/sshd_config, but it is possible to change this using the -f switch on the command line used to start the server. Check your startup scripts to be sure.
  3. Audit the user accounts on your system. This is the sort of more general security measure that you should be taking anyway, but it’s worth mentioning here. Some common account names are targeted by the automated brute-force password cracker scripts out there, so be careful of accounts with names like admin, user, guest, test, etc. Where possible also make sure that the system accounts required by some services have no passwords set and have something sensible set as their shells like /bin/false.
  4. Think carefully about from where you need to allow access to the server: does every machine on the net really need to be able to connect to it? You can use the /etc/hosts.allow and /etc/hosts.deny files to control access (see man hosts_access), as well as setting up iptables rules only permit access from trusted IPs or ranges of IPs, for example (assuming you drop incoming packets by default):
    iptables -A INPUT -s $TRUSTED_IP -p tcp --dport 22 -j ACCEPT
  5. Carefully read the relevant manual pages, particularly sshd and sshd_config. Then read your /etc/ssh/sshd_config file – do not assume that your vendor or distribution has not altered any of the default variables. Some variables to pay particular attention to include:
    • Protocol – set this to 2 so as to not allow connections to fall back on protocol 1.
    • Make sure that PermitRootLogin in set to no.
    • Set PasswordAuthentiction to no whenever possible.
    • Set PubkeyAuthentication to yes (default) and set up keypairs for users (see man ssh-keygen and man ssh-agent).
    • Set PermitEmptyPasswords to no (default).
    • Set StrictModes to yes (default).
    • Set UsePrivilageSeparation to yes (default).

    Hopefully most of these variables will be in the states described. Note that some linux distributions use PAM to authenticate users connecting to the SSH server. This is beyond the scope of this article – see the UsePAM directive in the sshd_config manpage and the Linux-PAM website for more information on this.

  6. You can use the AllowUsers and AllowGroups directives and their DenyUsers and DenyGroups counterparts to limit access to trusted users or deny access to system accounts or untrusted users. Particularly useful if you need to allow password-based authentication.
  7. If you need to allow blanket access to the server from the internet, investigate methods for throttling large numbers of simultaneous incoming connections. Two worth investigating are:
    • The built-in MaxStartups directive in the sshd_config file. This sets the number of unauthenticated connections that are allowed at any one time before the server starts ignoring new connection attempts, and can be useful if you find your server is being targeted by brute force attackers making lots and lots of simultaneous or near-simultaneous connections. Read the sshd_config man page for more information on this.
    • Use the recent extension to iptables to track incoming connections and drop any from the same IP within a given number of seconds. This is very effective aganist the brute-force password crackers:
      iptables -A INPUT -p tcp -m state --state NEW --dport 22 \ 
       -m recent --update --seconds 15 -j DROP 
      
      iptables -A INPUT -p tcp -m state --state NEW --dport 22 \
       -m recent --set -j ACCEPT
  8. Some people may recommend moving the server to an unusual port by setting the Port directive in sshd_config, others may suggest implementing port knocking to control access. Although these solutions may afford you some additional security, the former is just a form of security through obscurity, although it’s probably quite effective against the automated scanners. The latter method is not without detractors, but can also be useful. Anyway, neither should be used as a substitute for other security measures, so I’ll say no more about them here.

This article has been re-written from a stub that really just referenced the article that inspired it, Securing access to your server checklist. Feedback is very welcome.