...making Linux just a little more fun!

May 2005 (#114):

The Mailbag

HELP WANTED : Article Ideas
Submit comments about articles, or articles themselves (after reading our guidelines) to The Editors of Linux Gazette, and technical answers and tips about Linux to The Answer Gang.

Python, tail -f, pyurlview, non-blocking I/O

Mon, 28 Mar 2005 05:03:50 +0100
Thomas Adam (thomas from edulinux.homeunix.org)


I'm hoping this will drum up some replies. I'm tumbling head-first into the world of python and ncurses for this, and know very little about both.

To cut a long (and rather boring) story short, I run irssi in a screen session on my poor P166 server. X11-forwarding is not an option as I don't have nor want X11 installed, and it's damn slow even if I did. But one useful feature is using one of the many scripts off of irssi's website, I can get irssi to scrape urls into a text file. "Wooooo, I hear you say".

Well, it is nice. And thanks to network transparency, I am able to tail that logfile via shfs (ssh filesystem, essentially). So then I got to thinking. What's an easy way of displaying the URLs in the file? I remembered a program that ships (often) with mutt -- "urlview". It scans a file for a regexp for URLs and presents a means of selecting them and launching them. Brilliant. Except... it expects a file -- piping to stdin blocks.

Bah! I don't want to hack the source-code, unblocked I/O in C is a headache. So I stumbled across pyurlview [1] a replacement written in Python. It's "ok" and is in a language I can read.

What I'm wanting to do is emulate this:

tail -f ./some_file | pyurlview

Now that doesn't work as-is. So I thought about it a bit, and came up with this little bit of python code (which I hope is the correct "way" to do it):

#! /usr/bin/env python
import os,sys

while 1:
  s = sys.stdin.readline()
  print "Got", s

Which seems to work when one does:

tail -f ./foo | testfile.py

So I thought I'd try and incorporate that idea into the pyurlview script [1] I tried all I might, to no avail. The closest I got was here [2] but that's mangling everything and isn't functioning like in the example I correctly showed above.

So if someone can see the error in logic, do say. Or does anyone know of an existing program that will do what I want? I can use any form of scripting language to do it (awk, ruby, etc) already -- but it's more the curses and non-blocking I/O I'm interested about. Plus, I'd rather not reinvent the wheel, hence battling it out with python. maybe I could use 'multitail' but it's UI sucks for what I am wanting.

Thanks In Advance,

-- Thomas Adam

[1] http://www.dgp.toronto.edu/~mac/code/projects/pyurlview.py

[2] http://edulinux.homeunix.org/~n6tadam/pyurlview-thomas.py

Which flavour?!

Tue, 8 Mar 2005 15:46:26 +0200
Gavin Kehlert (GavinK from LechabileSS.co.za)

Hi Guru's.

I'm a novice as far as Linux goes and want to attend a course but don't know which flavour to go for or who supplies the best training. From what I have seen my choice should be between RedHat and SuSE..

Any suggestions?

Thanks in advance,


Re: Possible correction to Introduction to Shell Scripting -

Wed, 6 Jul 2005 11:42:31 -0400
Benjamin A. Okopnik (editor from linuxgazette.net)
Question by peter.yellman (peter.yellman from webformation.com)

[ cc'd to The Answer Gang ]

Hi, Peter -

On Sat, Apr 09, 2005 at 11:53:18AM +0000, peter.yellman wrote: At http://linuxgazette.net/113/okopnik.html.

When it comes to shell scripting I am a rank novice, but it seemed to me that the passage "if $1 has a length of zero, then the following statements (echo... echo... exit) should be executed" (quotes not mine) should read "if $1 does not have a length of zero, then the following statements (echo... echo... exit) should be executed"

In fact, the article is correct - the

[ -z "$1" ] && { ... }

construct will execute the statement block if $1 is zero length; the "-z" operator returns "true" in that case. You can try it on the command line:

ben@Fenrir:~$ foo=abcdefg     # Give some value to 'foo'
ben@Fenrir:~$ [ -z "$foo" ] && echo "Foo is empty"
ben@Fenrir:~$ foo=     # Unset 'foo'
ben@Fenrir:~$ [ -z "$foo" ] && echo "Foo is empty"
Foo is empty

Thanks for the article,

You're welcome - glad you're enjoying it and thinking about it! :)

Re: [SEAPY] PyCon article

Fri, 8 Apr 2005 17:43:43 -0700 (MST)
Mike Orr (LG Contributing Editor)
Question by A group of Python users in Seattle (seattle-python from lists.seapig.org)

Dan Everhart wrote:

mso@oz.net wrote:

My PyCon 2005 article is up. Newbies will have to bear with some unexplained vocabulary (e.g., Lucerne, which is a full-text indexing system).

In case somebody wants to google, the correct spelling is "Lucene".

No wonder I couldn't find pyLucerne on Google!!!

-- Mike Orr

Speech Recognition (Not Voice)

Thu, 28 Apr 2005 11:04:30 -0400
jkinz (jkinz from kinz.org)

Hi Gang,

Just a quick note about the difference between "Voice recognition" and "Speech recognition".

Voice recognition means recognizing "The person who is speaking". This is a means of identifying people/persons by listening to their voice. We all do this whenever we use the phone and know who is on the other end as soon as they start talking.

Speech recognition means recognizing the words being spoken.

Voice recognition tools are used to identify people as part of access security systems.

Speech recognition tools (Dragon Naturally Speaking, ViaVoice) are used to "speak" into a computer microphone and have that computer understand what words you said. Currently Naturally Speaking seems to hold the edge in recognition accuracy. This can vary from person to person.

Janine's articles, [issue 113] Voice recognition shorthand and the birth of Weblish and [issue 87] Linux-Based Voice Recognition are actually about Speech recognition, not voice recognition.

-- Jeff Kinz, Emergent Research, Hudson, MA. (A former Speech Recognition software developer)

Thanks for clarifying the distinction, Jeff; we'll pass this on to our readers. I suspect that the common usage isn't likely to change much as a result - but, to misquote Larry Wall, I do occasionally struggle feebly against ignorance, and I admire other people's efforts in that direction as well. -- Ben


palmdoc problem

Sun, 1 May 2005 20:26:50 -0400
Wong Louis (mylover01ca from yahoo.com)

Hi, I am using debian unstable with firefox 1.0.2, I try to use the http://linuxgazette.net/cgi-bin/TWDT.pdb and put in the issue number. but it always d/l issue 113 no matter what I put in. I want to d/l the older issues in palmdoc format, how do I do it? Thanks

[Ben] Hi, Louis -
Well, being The Nice Guy that I am :), I've just tweaked the chunk of CGI that generates that file to be a bit more flexible; what you're asking for is now available. You can grab any past issue that contains a TWDT.html in PalmDoc format by going to
where NAME is the name of the issue. Do note, however, that
  1. Not all of the past LG issues have a TWDT.html (which is what's used to generate the PDB doc.)
  2. Issues before #100 are actually called "issueXX", where 'XX' is a two-digit issue number (i.e., leading zeros for low numbers.)
To make it simpler, you can use any sort of a bogus string for NAME, and you'll get a list of all issues with links that will return a PDB for that issue. I hope that helps.

This page edited and maintained by the Editors of Linux Gazette
HTML script maintained by Heather Stern of Starshine Technical Services, http://www.starshine.org/

Published in Issue 114 of Linux Gazette, May 2005

The Answer Gang

Linux Gazette 114: The Answer Gang (TWDT) The Answer Gang 114:
...making Linux just a little more fun!
(?) The Answer Gang (!)
By Jim Dennis, Jason Creighton, Chris G, Karl-Heinz, and... (meet the Gang) ... the Editors of Linux Gazette... and You!

We have guidelines for asking and answering questions. Linux questions only, please.
We make no guarantees about answers, but you can be anonymous on request.
See also: The Answer Gang's Knowledge Base and the LG Search Engine


¶: Greetings From Heather Stern
(?)How do I get a non-US keyboard mapping in Xnest?
(?)Question about file mtime on linux
(?)Xlib WMs and Applications
(?)icewm light weight desktop

(¶) Greetings from Heather Stern

Greetings, and welcome to summertime in the world of Linux Gazette. As promised last month we've a few juicy bits for you; we hope you like them.

Got Questions?

We're seeing a drop in the number of good questions though. Or even bad ones. Jim (the Answer Guy himself) attributes this to all the good questions having already seen answers in past columns. This can't be completely the case - and on rare occasions the answers do change and why they had to is a juciy discussion. But there's certainly some grain of truth to it.

We like it when more people ask the answer gang their linuxy questions.

Ah, but that's not the topic I've planned to yak about for this month. Maybe in a later column, when The Answer Guy brings back his Retrospectives, I'll do a nice breakdown on how people find helpful friends these days. But I've been having a lot of fun (and even some clients) doing web glitz, so I'm probably going to talk about that next month. Send in your own tidbits of good web techniques, we'll make a pool of them.

Good Companions

I had spring cleaning last month, which means that this month there is room on my desk - and in my office - for some nifty things. Right now my lab's about to bloom with new power and color, as I bring machines from my internet lounge into gear. I haven't decided if I'll run them with full distros this time, a liveCD again as I usually do (Knoppix being the front runner), or as terminals served off one higher quality machine, in the same spirit that the K-12 LTSP people do. One way or another it's bound to be a fun time at Baycon again this year, as my starship gets a Piece O' Da Action. I should invite our Editor In Dark Glasses along, and see if the ladies who fly the black helicopters are inclined to TAG along...

My teddybear now has a permanent place of honor on my FVWM desktop. If you've been paying attention you've got a copy of him in your /usr/share/xteddy directory too, from our February cover art and perhaps he brightens your day. Thomas gave me a rather sweet birthday present that I hadn't gotten around to showing you folks - note, this is very specific to FVWM, and you have to name your bear correctly, too.

First, you symlink xteddy to the name bear so the window title will be seperate.

Second, add this fragment to your .fvwm2rc. It's kind of important to advise FVWM that he accepts WM hints:

See attached misc/tag/bouncing-bear.fvwm.txt

Third, I added a little bit to my fvwm's StartFunction which makes sure that bear's on my desktop. You'll probably have the geometry a little different - have fun with it...

AddToFunc StartFunction
+ I Test (init) Exec exec /usr/bin/fvwm-root /home/heather/.fvwm/stars16a.xpm
+ I Test (init) Exec exec bear -wm -geometry +1320+820

Now anytime I press Shift-Alt-B he bounces. The only mystery I haven't solved - feel free to send in if you're a fvwm fan who knows the answer - is that if I have clicked him, say to drag him out of the way momentarily - then he won't bounce anymore, until I've minimized him. then his minimized form bounces fine, and so does his restored form. Pretty silly seeing the word with no picture bounce like that. It makes me giggle.

I hope you're enjoying a happy Summer. As for me, it's too bright out there, time to go hide in my starfields for awhile. Stargazing fans, the skies are clear, and there's plenty of software to tell us what to look for. In the realm of spaceward travels, things are... if you'll pardon the expression... looking up. Bye now!

You, too, can join The Answer Gang. Send your submissions to: tag .at. linuxgazette.net - Thanks!

(?) How do I get a non-US keyboard mapping in Xnest?

From Matthias Arndt

Answered By: Thomas Adam

Dear Answer gang,

it has been more than 4 years that I am reading (and sometimes publishing in) the Linux Gazette with real pleasure. I always did find most answers to my questions here but this time I'm really stuck.

From time to time I use the Xnest command to connect to remote boxes with graphical login.

This works much as expected except for one thing: I'm used to a german keyboard layout and my local X server is configured to use it. But Xnest does not use it - I always get US keyboard layout inside Xnest. For example I tried:

$ Xnest :1 -broadcast -xkbmap de -geometry 980x680
Cannot open "keymap/de" for reading
(EE) Error opening keymap file de, reverting to defaults

for example but I got above error and still only got a US keyboard inside. The rest was working as expected. Another try was something like this:

$ Xnest :1 -broadcast -xkbmap /etc/X11/xkb/keymap/xfree86 -geometry 980x680

Cannot open "keymap/etc/X11/xkb/keymap/xfree86" for reading
The XKEYBOARD keymap compiler (xkbcomp) reports:
 > Error:            Cannot open
"/var/tmp/etc/X11/xkb/keymap/xfree86.xkm" to write keyboard description
 >                   Exiting
(EE) Couldn't load XKB keymap, falling back to pre-XKB keymap
(EE) Error opening keymap file /etc/X11/xkb/keymap/xfree86, reverting to

The main question is: how do I need to call/configure Xnest that it will use my german keyboard setup inside the Xnest window so that I can use remote applications with my german keyboard?

I'm using SuSE Linux 9.1 with XFree86 Version (4.4.0 RC 2) and the according version of Xnest.

(!) [Thomas] The problem you have, Matthias, is that Xnest does not support the -xkbmap inherently. Yes, although Xnest will emulate a server, at present Xnest is only capable of using its own default keyboard extension map.
You have to remember that Xnest is a complex program. To the Xserver and indeed any other Xclient, it looks and behaves as such (it looks like an Xserver to the underlying real Xserver, and as an Xclient to the windows mapped to it) [1]. The reason why the -xkbmap is ignored is because when Xnest loads, it changes the keyboard layout procedures of the real server to its own defaults [2]. Currently there is no pure workaround, other than you might be able to use Xmodmap from within Xnest to change it (or possibly Xkeycaps).
I realise I've been vague about some points, but this is a really complex issue that I might (if the interest is high enough) turn into an article or TAG entry.
  1. This has several inherent problems, that I'll skip.
  2. To many people, this is complex to work around -- but think about. Xnest is both an Xclient and an Xserver (in this sense). The kdb extensions are inherently supplied and then dropped by the real Xserver. It's a transparent process only.

(?) Question about file mtime on linux

From Suramya Tomar

Answered By: Benjamin Okopnik, Pete Jewell, John Karnes, Mike Orr, Jay R. Ashworth, Kapil Hari Paranjape

Hi Everyone, I have a question for you about file mtime(modify time) on linux. Does the mtime stamp of a file change as soon as a process starts modifying the file or does it change it after the change is done?

(!) [Ben] After the change is done, of course; up until that time, the file has only been read, not written to. What is modified (in e.g., an editor) is a copy of the file, generally held in memory - which is why the "save" function exists. Otherwise, a crash in the middle of editing would destroy your original file.

(?) The reason I am interested in this is that I am writing a perl script which is supposed to monitor a file for changes and as soon as the change is done run another script which processes the changed file. I don't want the second script to run before the file change is complete(Which would happen if the mtime changes as soon as the modification starts).

So how would I check that the file change is done?One way would be to keep checking in a loop if the mtime changed if it did goto sleep and check again after a few seconds, keep repeating this until the mtime stop changing. But in my opinion this is a stupid way of doing this so I am hoping one of you has a better way of doing it.

(!) [Kapil] File locking might be useful. The script sees the change in the mtime, then waits for the lock to go away and starts processing. You would have to tell the modifying program to use file locking though.
(!) [Sluggo] I would agree to use the kernel monitor first. But for alternatives...
Why not have the first process send the second process a signal when it's done? Put the observing process in indefinate sleep, and have the write process send a SIGALRM (Alarm) to kick it. You'd prob'ly want to put the observer's PID in a well-known file so the other process can find it.
Or use file locking. The writer holds a write lock till it's done. The observer sees the mtime change and acquires a read lock. The read lock blocks until the write lock has released.
If you really want to know when the mtime is set, look in the kernel source. or the libc source, and see what fwrite() does. That's one of the advantages of open-source software. Even if you don't know much C, you can still tell whether the word "mtime" appears above or below a write() call.
Or just think about why most programs don't have to worry about this. Unix programs tend to open-write-close quickly, and close the file when they don't need it. (As opposed to Windows text editors, which often hold the file open the whole time.) Unless the program has to hold the file open for a long time (e.g., streaming log entries), your chances of hitting the file in the middle of an update are pretty slim. Then think about, what's the worst that would happen if you did? Your observer would produce garbled output, with part of one version and part of another. Or maybe it would crash. Would this be the end of the world or a minor inconvenience? At least it would tell you how (in)frequently such a collision is occurring.
(!) [Pete] From past experience I seem to remember that the only 'pure perl' way to ensure that a file is not being modified, without relying on file locks, is to check the size/mtime of the file, wait a bit, and then check it again, repeat until the two size/mtimes are the same.
The reason I ran into this was because I was writing a routine to process a file after it had been uploaded to an ftp server. In the end, because we had control over the ftp server, we configured it so that the uploader and my routine both logged into the ftp server using the same username, and restricted the number of times a user could concurrently login to 1. This did the trick quite nicely.
A good resource for perl questions is the Perl Monks website http://www.perlmonks.org - in fact I would go so far as to say that it is the best resource for perl information on the web.
(!) [Ben] [grin] Multiple xterms are useful for this. In one of them, run this program:

See attached mike.show-mtime.pl.txt

where "foo" is the file you're looking at; this will print the mtime of 'foo' once a second. In another xterm, open 'foo' and modify to your heart's content - all the while glancing at the first term. Nice and easy.
(!) [John Karns] How about making a call to 'lsof' to see if the file is open?
I'm not sure if a latency in the kernel flushing disk buffers would be a concern in this kind of scenario. If so, you might want to have either one of the processes make a call to flush the buffers to ensure that there is not a pending update to the file.
In fact I've sometimes wondered about this myself: if there is a pending write to a file via a dirty buffer, is that automatically taken into account if I read the file before the buffer is flushed? I.e., is the pending change transparently mapped to a read of the file by libc or whatever?
(!) [Sluggo] It would be a very severe bug if it didn't read from the buffer, since that's the official version currently. I've sometimes wondered this myself, but I've found the kernel developers pretty trustworthy so I assumed they wouldn't do such a thing.
(!) [Jay] It seems to me that the kernel should update the mtime in the inode (as the inode is transparently cached in RAM) *everytime a write(1) call is made to the file*.
So, at this point, your question expands to "if someone makes a write call which takes a finite amount of realtime to execute (like, writing 1MB from a RAM buffer to a file), at which end of the execution of that write call will the inode get updated?"
IANAKH, but I believe the pertinent code is in kernel/fs/$FILESYS/file.c
and specifically the function ext2_file_write() (for ext2).
Short version:
  1. ext2 updates the inode after it writes the actual data,
  2. since it's buffered, the order in which the actual bytes hit the disk is indeterminate, and
  3. it's filesystem specific, since the answer to the question lives in a filesystem specific location in the source.
The specific answer lies in the relative positions of ll_rw_block() and the assignment to inode->i_mtime in the function.

(?) Wow, I didn't expect my question to generate such a big response. Now I know why I like TAG :)

NEways I did a little research after reading all your answers and found the following:

The file modification time changes continuously when
the file is being created/modified. (For example when
creating the file using cp).

To test this I wrote the following perl script that displays the mtime of my test file every 1 second:

See attached suramya.show-mtime.pl.txt

Then I opened another xterm window and ran the following command: 'cp /var/log/messages test.dat'

suramya@StarKnight:~/Temp$ perl test.pl
Time: 1113452801
Time: 1113452801
Time: 1113452801
Time: 1113452801
Time: 1113452801
Time: 1113452801
Time: 1113452801
Time: 1113452890  <-- Changed
Time: 1113452891  <-- Changed
Time: 1113452891
Time: 1113452893  <-- Changed
Time: 1113452894  <-- Changed
Time: 1113452895  <-- Changed
Time: 1113452896  <-- Changed

As you can see when the copy started the numbers started changing. So now we know that the mtime keeps changing when the file is being modified. And if you think about it, it makes sense: The mtime changes whenever any changes are made to the inode's used by the file so when the file is being created new inodes are being used constantly so the mtime has to change.

Now I will be looking into the other suggestions you guys made and see if I can get this to work. (And no this is not a school project  :) ) I need to export data from an oracle DB to a CSV file and have another script read this CSV file and process it. If the second script reads a half written file 'Bad Things' (TM) will happen.

(!) [Pete] If that's the case, then a low tech solution might suffice. Write your CSV file out from oracle as something like 'temp_output.csv', and then have your oracle process rename the file once it's fully exported. Then your perl script is only looking for the renamed file, instead of trying to guess when the file creation is complete.
(!) [Ben] Oh, nice solution, Pete! You could even follow the (more or less) standard practice of "building" the file in a temp dir, then moving it into the location from which it should be copied. As long as the temp and the final locations are on the same partition, that's just a matter of changing the inode info - which is an atomic op, so there's no chance of retrieving a partial file.

(?) Thanks a lot. This solution works perfectly for my task. Makes my life a lot simpler too :)

(?) Xlib WMs and Applications

From Kapil Hari Paranjape

Answered By: Thomas Adam

This reply is a followup to http://linuxgazette.net/issue114/tag/4.html but is more specific to X11 itself. -- Thomas Adam

But I'm often confused as to what "the other half does". For example, I am now using "fvwm" (thanks to Thomas' suggestions on how to ratpoison-ify it). I know that "fvwm" traps some keys and uses them for its own nefarious purposes.

(!) [Thomas] Key-bindings as set by the user for the window manager are always greedy with respect to the window manager. So for instance, in FVWM, one might have a key-binding such as:
Key X   A 3   Function FvwmTitleRxvt
Now, of course, it might be the case that windowskey+X is an actual binding to an application I have running, but it's tough -- the WM has precedence in that binding as I have it defined there.

(?) What happens when it is used with GNOME? Who is gets the first option to look at the keys or other input events? GNOME or fvwm?

(!) [Thomas] Always the window manager. Remember that GNOME is just a framework and a collection of utilities. If you wanted a hierarchy, a depiction such as the following is an accurate representation:
GNOME <--> window manager --> Application
Of course, the integration aspect of GNOME <--> Window Manager how depends on EWMH. I've talked about these in the past, but I might as well summarise them here for clarity. EWMHs are extensions from window states which were first outlined by the ICCCM manual [1]. Unfortunately, the ICCCM is rather old, but it does provide a fantastic framework on which to build a WM, and it is still very relevant today. When KDE and GNOME were developing into the bloatware we see today, one thing they both did were to extend the ICCCM to define their own window states. Of course, compatability became a nightmare in such instances, and so these were cleaned up and standardised by the freedesktop people [2]. They're only windowing hints at the end of the day for things such as 'Working areas', 'Number of Desktops', and such like.
You may ask why the WM has precedence over an application, when it is the job of the WM to manage applications. The answer is simple. Key-bindings are the job of the Xserver. So actually the propogation looks like this:
Xserver -> Window Manager -> Application.
When you get your window manager to bind your keys for you, what's actually happening is that the window manager will map the requests itself, so it keeps a record of WM-specific keybindings. When the WM grabs the Xserver when it maps an application or a key-binding it then checks to see (via the Xserver) whether that keybinding is WM-defined or not. If it is then it will act appropriately, else it will pass it down to the underlying application, or do nothing.

(?) A similar question could be raised regarding how window positioning and decorations are decided/executed. Perhaps the overall GNOME/GTK theme decides and passes on the info to the window manager which actually executes these?

(!) [Thomas] Not quite. GTK itself is window manager agnostic. When you apply a GTK-theme, all that happens is that any currently running GTK apps are redrawn by the Xserver. There is no interaction with the window manager in anyway while this is happening.
Of course, GTK themes don't really allow for the stylation of title bars -- that's the job of the WM, and rightly so. You have to remember thatit's the job of the xserver to map the window which the WM then grabs and decorates as it sees fit. So the decoration could come in any form.
Window positioning is a separate entity in itself. When windows are mapped to the Xserver, the server will naturally map it to 0+0 which is top-left. The window manager already has one window that is top-level, and that is the root-window. This is the ultimate window from which everything else stems from, when windows are created. Consider the following Xlib snippet:
win = XCreateSimpleWindow(dpy, DefaultRootWindow(dpy), xsh.x, xsh.y,
	xsh.width, xsh.height, bw, bd, bg);
I could easily spend hours talking about this line of Xlib alone, but I'll save you the gory details. Essentially, the point you should realise is that all windows created in this way [3] are always a child of the root-window. Why? Well, it's all about management. Once the Wm knows the top-level window, it knows how to stack windows as well as place hints on them. Of course, the WM is not allowed to change application-set windowing hints when the window is being mapped. Indeed, this is really where 'xprop(1)' comes in useful. This allows one to examine all of the window states as well as set them. There's also 'xwininfo(1)' for a more high-level view, although to be honest, I prefer FvwmIdent for such things.
Of course, with window hierarchies comes the notion of Transient windows. These are (by definition) short-lived windows, such as dialog boxes. They'll often have the the following hint set:
[n6tadam@station ~]$ xprop | grep -i trans
WM_TRANSIENT_FOR(WINDOW): window id # 0x2a0002e
The window manager might well treat these differently -- indeed, FVWM does. Due to their short-lived nature, it's often not desireable to have all buttons on the titlebar, so FVWM might well not decorate transient windows in such a manner. Of course, one can change that with:
Style * DecorateTransient
Anyway, window positioning is managed by the window manager. The geometry of a window defines its position on screen, and hence its size. In the form:
widthx height+/- xorigin+/- yorigin
Note that the window geometry will often be with respect the actual window itself, and not the frame that surrounds it. In slight relation to that is a window gravity. There's 'Window Gravity' and 'Bit Gravity', and they specify for the window as a whole (including the borders) and the window contents how to react on window resizing.
Of course, as you have no doubt guessed, it is not quite as simple as -geometry. There is certain other hints that provide additional positioning. Take for instance Mozilla. That doesn't have a '-geometry' option because of they way it was designed. Instead that remembers its position (and hence window size) each time it is closed. The WM can be instructed to ignore or react to such things. As is the case with FVWM there's a hint called 'NoPPosition'. This instructs FVWM to place the window based on what/where it tells the WM to put it. Hence:
Style gecko NoPPosition
would be such an example [4]. There are instances though where a window will use PPosition hints as well as accept geometries. Notably, [X]emacs is such a beast and will set its PPosition to (0,0) each time, so overriding that is paramount for some.  :) Again, geometry specifications take precedence over PPosition hints where they're supported.

(?) A pointer to relevant sources of information would be greatly appreciated.

(!) [Thomas] I hope this helps. I could rabbit on for ages about all of this, but I'd rather answer such things as they come up, as opposed to overloading you with tedious information. Unfortunately, information such as this is sparse, and represents my understanding through working with FVWM, and Xlib in general.
  1. http://tronche.com/gui/x/icccm
  2. http://standards.freedesktop.org/wm-spec/wm-spec-1.3.html
  3. Of course, it doesn't have to be at the low-level such as Xlib. Often, languages have wrapper functions around XCreateSimpleWindow() which will automatically map a window to be a child of the root window.
  4. http://fvwm.lair.be/viewtopic.php?t=273

Dear Thomas,

Thanks for the detailed write-up. Any chance that this will one day become an article somewhere? (Hint! Hint!).

Great stuff.

This question was raised about a specific window flag. I've split it out from the above commentry to keep readability intact. -- Thomas Adam
(!) [Thomas] Of course, as you have no doubt guessed, it is not quite as simple as -geometry. There is certain other hints that provide additional positioning. Take for instance Mozilla. That doesn't have a '-geometry' option because of they way it was designed. Instead that remembers its position (and hence window size) each time it is closed.
(!) [Ben] Oh, how I wish it did.  :) Not a big thing, but this has been a long-standing problem with Mozilla for me (both the Big Moz and now Firefox): Due to couple of info bars that I use (MailDooHicky and the Moaning Goat Meter),I like for my windows to span from the left edge of the screen to ~100px short of the right edge, and from ~30px down from the top to ~30px short of the bottom. Xterms are, of course, no problem - I've got "XTerm*VT100*geometry: 134x40+0+21" in my ~/.Xresources, and that takes care of it. Mozilla, well... I've got a little Java bookmarklet on the toolbar that makes it jump to the correct position whenever I find it too annoying. But it never remembers the position.
(!) [Thomas] It does work, but it's dependant on how the window manager deals with such things. I have to say that some WMs are better than others in this regard. It has been many years since I used icewm for any serious use, but as far as I can recall, it doesn't allow for arbitrary commands to be run on a window.
(!) [Ben]
Eh, well... for certain values of 'arbitrary'... kinda sorta. :) You can't (so far as I know, at least) run commands on a window once it's up. You can, however, assign initial properties to windows (actually, your discussion of all this stuff is what I needed to remind me of that - thanks!) by defining them in ~/.icewm/winoptions; in this case, what I needed was:
Gecko.Firefox-bin.geometry: 1212x722+0+23
This sets the initial position of the window to what I need, which is perfect.
(!) [Thomas] I hate to err, draw comparisons.
(!) [Ben] No you don't.  :) C'mon, you love to tout the superiority of FVWM! (And why not? If it can do something better than everybody else, go for it. After all, it's not boasting if you can do it.)
(!) [Thomas] In FVWM, doing what you're describing above can be achieved in two ways, the first is to use a function, as in:
DestroyFunc FuncStartMoveBrowser
AddToFunc   FuncStartMoveBrowser
+ I Exec exec $0
+ I Wait $0
+ I Next ($0) ResizeMove w+0 -100px 0 -30p
(I've most likely got the co-ordinates wrong, but that can be left as an exercise to the reader).
So the above will exec the first formal argument to the function ($0), wait for it to appear, and then will move and resize it to the said co-ordinates. You might invoke it thus:
FuncStartMoveBrowser mozilla
One would replace 'mozila' with whatever program was wanted. But there are limitations with that method. The first one is that it will only work when invoked from within a menu, or FvwmConsole, and won't work if one were to type in 'mozilla' from an Xterm. In such cases where this is more desirable, FvwmEvent can be used, as in:
DestroyModuleConfig FvwmEvent-Browser: *
*FvwmEvent-Browser: Cmd Function
*FvwmEvent-Browser: add_window  FuncStartMoveBrowser
Which would set up an event for each window that was created, calling the function specified. The function as we have it defined needs modifying for use with FvwmEvent, as the window would have already been created, hence:
DestroyFunc FuncStartMoveBrowser
AddToFunc   FuncStartMoveBrowser
+ I All (CurrentDesk, AcceptsFocus, !Transient) ThisWindow ("gecko") ResizeMove w+0 -100px 0 -30p
I've used 'gecko' as that is the window class common to both Mozilla and Firefox.
You're probably wondering how this helps you, Ben. It doesn't, but there is an application which can move and resize windows which you could make use of, coupled with xwininfo(1). It's called 'xwit' and is just a wrapper program around some of the Xlib functions, so for example:
xwit -move x y -resize x y -names 'mozilla'
If you wanted to get specific, you could use xwininfo to fine tune things, matching by window ID, etc. Maybe a little crude, but it might help you.
(!) [Ben] [laugh] Well... it's got this little problem built in...
ben@Fenrir:~$ xwininfo|grep 'Window id'
xwininfo: Window id: 0xc00031 "Google Advanced Search - Mozilla Firefox"
Since Firefox changes its window name to whatever the site happens to be, I have to click the Firefox window in order to find out its window ID (which could then be used to do 'xwit' twiddling.) This is, shall we say, suboptimal for the task.  :) I'd imagine there are other ways to hunt it down - 'xwininfo -all -root|grep Mozilla' comes to mind - but that gets a bit shaky. Anyway, you've avalanched some synapses for me, the appropriate brain cells fired, and All Is Now Well.  :) Thanks!
(!) [Thomas] No worries. You can probably get a fairly accurate result in making use of '-root' to xwininfo, so something like:
[n6tadam@station ~]$ xwininfo -root -all -tree | grep -m1 'Mozilla-bin'
     0x282c250 "Gecko": ("Gecko" "Mozilla-bin")  200x200+0+0  +0+0
At least with it displayed in a tree form (and stopping at the first match), you can be fairly well assurred that you'll be matching the parent window.

(?) icewm light weight desktop

From Jpydeep Bakshi

Answered By: Chris Gianakopoulos, Benjamin Okopnik, Adam Engel, Thomas Adam

Hi all,

I am totally shifted to icewm and I am a newbie on this icewm. that's why I am asking some questions, may be very basic to you all but I don't have much knowledge about it.

1) is it possible to get a customised menu where I'll have some very frequently accessed entries like sylpheed-claws, mplayer etc and nothing else ?

2) how to create shortcuts on desktop ?

(!) [Thomas] Use a program such as rox-filer [1] or 'idesk' (search LG.net for that one, there was an article about it not so long ago). You're also suffering from a fundamental lack of understanding about what a desktop environment is, and what a window manager is. I'm not going to cover it here again - see any of the recent LG releases within the last six months. Essentially the term "desktop" isn't. It's just the root-window.
  1. http://rox.sf.net

(?) I'll definitely go through those article. what window manager do you use?

Now a million-dollar question. is it practically possible to run K-desktop environment on xfce or icewm window manager ? if possible then how ? I am too much interested to do such an experiment.

(!) [Thomas] You clearly don't pay enough attention to reading the LG, if at all. :) [2,3]. I suppose you ought to read the recent TAG entries as well [4,5,6].
Only where those window managers are EWMH compliant [3], for which icewm seems to be. This allows things like the kde kicker and gnome's taskbar to be handled correctly from the window manager that will run 'inside' of the KDE or GNOME environment. Effectively this process is using Session Management [2], although in reading that, you would need to replace the current running window manager in GNOME with icewm. You can't use "icewm --replace" as that's specific to FVWM, hence you should use:
killall metacity && sleep 1 && icewm &
Then you'd have to save your session before you logged out (via gnome-session-properties). I have to say that I absolutely hate session managers. I can't stand them. They're terrible at what they do and what they do, is second rate.
If you want to use KDE instead, the process can get a bit trickier - I have heard rumours of the use of candles and goats, but....
KDE is started via the 'startkde' script. If you edit that script what you might see is a line such as:
which starts the window manager (kwin by default). To start fvwm you should add the option "-w icewm" to the ksmserver line, and save the file. On typing 'startkde' this should then launch icewm as the window manager of choice.
Of course, all of this is theory - I don't use icewm so I am just applying what I already know.
  1. http://linuxgazette.net/105/adam.html
  2. http://linuxgazette.net/100/adam.html
  3. http://linuxgazette.net/104/tag/4.html
  4. http://linuxgazette.net/105/tag/1.html
  5. http://linuxgazette.net/105/tag/2.html
(!) [Ben] Thomas already mentioned that you're suffering from a bit of confusion in this regard. You don't "create shortcuts on desktop"; you run an application that allows you to create clickable icons linked to applications. My favorite is "coolicon", which is available as a standard Debian package; it does not come with particularly exciting icons - but you can use any XPM image as an icon, and it's very flexible, powerful, and lightweight.
You can also create clickable links on the IceWM toolbar - here's my ~/.icewm/toolbar file as an example:
prog "BackFlip" "xv.xpm" bkgr
prog "Lock screen" "lock_16x16.xpm" xlock -random
prog "CD Player" "dvd_unmount_16x16.xpm" grip --small
prog "Mix" "music_32ndnote_16x16.xpm" /usr/bin/tkmixer
prog "Firefox" "mozilla_16x16.xpm" /usr/bin/firefox
prog "JPilot" "pda_black_16x16.xpm" /usr/bin/jpilot
# prog "Liferea" "rebuild_16x16.xpm" /usr/local/bin/liferea
prog "Snap" "camera_16x16.xpm" /usr/local/bin/snap
prog "Cell-up" "randr_16x16.xpm" /usr/local/bin/cell-up
The first string after "prog" is the 'hover label' for the icon, the second one is the 16x16 XPM image to be used, and the third one is the program to execute. Simple and easy.
Furthermore, IceWM allows you to associate programs with other toolbar entities (the mail notification icon, the CPU activity icon, the clock, etc.) See the "preferences" file - same locations as the "menu" file - and search for the word 'Command' (the variables will be MailCommand, ClockCommand, etc.)

(?) 3) how to add wallpaper ?

(!) [Ben] You can use "icewmbg", or just use the standard X commands (i.e., "xsetbg").

(?) 4) can I add application-button, applet to panel which I could do in KDE ?

(!) [Ben] I think you're asking about toolbar buttons - which I've answered above.
(!) [Chris] I'm an icewm boy. I can answer two of your questions. In your home directory, once you save your preferences with the program called 'icepref', you will have a directory called .icewm. There will be a file called preferences.
For a background image (wallpaper), set the following line:
In this case, a jpeg file will represent your wallpaper.
I have attached another file in that directory called programs. It represents your menu items. The file that I have included is the one that I use. Modify this as you like. In it you will see menu entries, and within each menu entry, you will see prog entries. The prog entries are the selections within each menu.
Actually, I have attached both (preferences and programs) files. If you don't have icepref, just create the directory .icewm in your home directory, and copy the attached files into that new directory.

See attached icewm-preferences.txt

See attached icewm-programs.txt

(!) [Adam] If you're using a Desktop Environment like KDE or GNOME, why do you need a win manager at all, except to literally "manage windows;" that is, what's the difference between Metacity or Icewm or Sawfish if you're already using GNOME with all it's icons, menus, etc.
(!) [Thomas] Because KDEs and GNOMEs framework are such that you can use different window managers within them as long as they're EWMH-compliant. It just so happens that GNOME uses metacity now as its window manager or choice. Before that, it was sawfish.
(!) [Adam] As "stand-alone" win managers, sawfish and metacity don't seem to do much (on my system) -- as compared to window maker or enlightentment or fvwm etc -- yet sawfish and metacity are the "window managers" of GNOME, which doesn't, as far as I know, have a win manager of its own.
(!) [Thomas] That's right, it doesn't. It's allowed for the choice to be down to the user. Of course, most don't bother changing it, because the options that GNOME gives them aren't dependant on the window manager being used, but rather how GTK will react to those events; in the same way that the options GNOME allows for affect the gpanel, etc.
(!) [Adam] I don't understand the relationship of a window manager, such as sawfish, to a full desktop environment such as GNOME. If I want to go with a window manager, I use window maker or enlightenment and customize it, otherwise, I use KDE or GNOME for the "desktop experience." Can anybody explain what the relationship between a window manager and a Desktop Environment like GNOME?
(!) [Thomas] See above -- it's only one half of the equation.

This page edited and maintained by the Editors of Linux Gazette
Copyright © its authors, 2005
Published in issue 114 of Linux Gazette May 2005
HTML script maintained by Heather Stern of Starshine Technical Services, http://www.starshine.org/

Published in Issue 114 of Linux Gazette, May 2005

News Bytes

By Michael Conry

News Bytes


Selected and formatted by Michael Conry

Submitters, send your News Bytes items in PLAIN TEXT format. Other formats may be rejected without reading. You have been warned! A one- or two-paragraph summary plus URL gets you a better announcement than an entire press release. Submit items to bytes@lists.linuxgazette.net

Legislation and More Legislation

 Software Patents

Following the European Council of Ministers' decision to discard Parliament's amendments, the legislative process continues. The FFII has reported the publication of the report by Michel Rocard, European Parliament rapporteur, on software patents. The findings and recommendations in the report have been welcomed by FFII president Hartmut Pilch. FFII has also published a recording and some transcripts from the JURI meeting at which the rapporteur's findings were presented.

Although, currently, campaigning for the European Constitution referenda is dominating European politics at the moment, the importance of contacting your MEPs on this matter should not be underestimated. Only by paying attention to their actions, and calling them to account for them (whatever your personal views), can citizens begin to reduce the democratic deficit manifest in EU institutions.


gpl-violations.org has been granted a preliminary injunction against Fortinet UK Ltd., a company that produces and distributes firewalls. Granted by the Munich district court, the injunction follows claims by gpl-violations.org that Fortinet have included various GPL software, including the Linux kernel, into their firewall offerings. Harald Welte, the Linux kernel developer involved in this case, also claims that Fortinet attempted to conceal their appropriate of the relevant code by using cryptographic techniques. Fortinet have admitted that GPL code was included in their product, but have so far failed to satisfactorily resolve the issue.

Linux Links

Using Python on the Nokia S60 mobile phone.

Userspace filesystem encryption with EncFS.

O'Reilly article on Sound editing with Audacity, and meanwhile Linux Journal has looked at Csound.

Comparison of Microsoft Visual Studio and KDevelop.

Ongoing history of open-source software by Peter H. Salus.

Are free-software developers the glimmer of hope for software quality?

Creating a custom Linux LiveCD With PCLinuxOS.

Linux Journal has published A Reading List for Linux in the Classroom, covering topics like OpenLDAP, Samba, and more.

IBM developerWorks has an interesting article that discusses running GNU/Linux or NetBSD on Apple's new Mac mini.

Groklaw's Pamela Jones argues that the SCO case has been good for Linux

Slashdot discussion of Linux laptop compatibility, while OSnews has Linux on A Laptop: A Tinkerer's Journal.

NewsForge has published a brief review of the UseMod Wiki.

The challenges of analysing GNU/Linux use.

The maturation of 64bit free OSs.

Has acquisition by Novell hurt SuSE on its German home turf.

Volvo are now doing crash simulations on GNU/Linux, in partnership with IBM.

Lawrence Lessig tells Flash developers how how he believes their work and technology is perceived in the open-source community.

News in General

 Who's The Git?

Over the years we've seen a number of heated arguments among the members of the Linux kernel project. Favourites among these have included the debates over preemption strategies and discussions of how to manage memory most effectively. A repeated focus for acrimonious discussion has been Linus's choice of source-management tool: BitKeeper. The selection of a proprietary piece of software for this fundamental part of the development infrastructure has rankled many of the more idealistic open-source/free-software purists in the development community.

This issue came to major prominence in the past month, as Andrew Tridgell publicly revealed that he had reverse engineered the BitKeeper protocol to allow developers to obtain project meta-data from the repository. On seeing this, the lead-developer of BitKeeper, Larry McVoy, responded by revoking the free BitKeeper licences that have been used by kernel developers since the tool was adopted by the project.

This sequence of events drew the ire of Linus Torvalds, who accused Tridgell of acting irresponsibly and jeopardising an effective and working system without offering any alternative. Others in the free software community, such as Bruce Perens, see these criticisms as unjustified, and point out that Tridgell was essentially engaged in the same type of activity that allowed him to develop Samba.

Nonetheless, in spite of these travails, Linux kernel development has not been stopped in its tracks. An alternative tool, git, to manage the kernel source (advice for beginners). Following the successful import of kernel source into git, the 2.6.12rc3 release is the first to be built up completely using git.

An interesting aside to consider is the response of the Subversion project to discussion among users that Subversion might be a suitable BitKeeeper replacement. Highlighting the difficulties in achieving a compromise between Linux development practice and the current Subversion feature-set, attention was instead briefly focussed on three new version control systems that have emerged in the past year or two Monotone GNU Arch and SVK. All three of these support the use of distributed repositories, and have recently been gaining popularity.

 Linux Kernel

The newest iteration of the stable 2.6.x series of Linux Kernels is now available. Linux, released on April 7th, includes a collection of fixes. In line with the new kernel numbering scheme, post 2.6.11, this release should only include minor fixes to the original 2.6.11 release.


Debian Weekly News has reported the availability of a HURD Live CD. This is a great opportunity to try out the long (long!) awaited HURD kernel.

Distro News


Archie is a complete live Arch Linux system (v0.7) that can be run from a CD/USB. OSDir has a screen-shot tour of the distro.


Debian stable (Woody) has been updated with the release of Debian GNU/Linux 3.0r5. The complete list of accepted/rejected packages in this release can be seen at http://people.debian.org/~joey/3.0r5/

Branden Robinson has been elected as the new leader of the Debian Project. He has given an interview where he outlines some of his plans.

From Debian Weekly News, Hanna Wallach has written an essay about making Debian a friendlier place for both men and women. Apparently, a significant number of men participate in the Debian Women project because it offers of a much more positive, welcoming, and friendly atmosphere than other Debian fora.


LinuxTimes.net has a review of Kanotix, a Debian Sid based live CD.


OSnews has a review of Linspire. Linspire is a GNU/Linux distribution aimed at providing an easy-to-use environment for users migrating from Windows.


Mad Penguin has taken a look at SuSE 9.3 Professional.


Ubuntu has released a new edition of its popular, Debian-based, GNU/Linux distribution. Ubuntu 5.04, (The Hoary Hedgehog Release) is in the wild and ready for download. In parallel with this, The Kubuntu project has released Kubuntu 5.04 (Ubuntu combined with the latest KDE goodness).

Linux Journal has recently taken a look at this rapidly growing distro.

Software and Product News


The Apache Project has announced the release of the newest version of its omnipresent web-server software. Apache HTTP Server 2.0.54, now available for download, is principally a bug fix release.


Those more familiar with the interface used by Blender, the popular open-source 3D modelling package may be interested in instinctive Blender. This fork has been profiled at NewsForge.

 Mastering *BSD Security

Not directly Linux related, but surely of interest to most Unix-like system admins (or should that be admins of Unix-like systems!), O'Reilly has recently released a new title focussing on security issues and techniques on BSD systems: Mastering FreeBSD and OpenBSD Security.

Mick is LG's News Bytes Editor.

[Picture] Originally hailing from Ireland, Michael is currently living in Baden, Switzerland. There he works with ABB Corporate Research as a Marie-Curie fellow, developing software for the simulation and design of electrical power-systems equipment.

Before this, Michael worked as a lecturer in the Department of Mechanical Engineering, University College Dublin; the same institution that awarded him his PhD. The topic of this PhD research was the use of Lamb waves in nondestructive testing. GNU/Linux has been very useful in his past work, and Michael has a strong interest in applying free software solutions to other problems in engineering.

Copyright © 2005, Michael Conry. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

Python for scientific use. Part I: Data Visualization

By Anders Andreasen

Motivation and outline

A first step towards qualitative understanding and interpretation of scientific data is visualization of the data. Also, in order to reach a quantitative understanding, the data needs to be analyzed, e.g. by fitting a physical model to the data. The raw data may also require some initial processing in order to become useful, e.g. filtering, scaling, calibration etc.

Several open source programs for data analysis and visualization exist: gnuplot, grace, octave, R, and scigraphica. Each of these has its own pros and cons. However, it seems like you always end up using more than one program to cover all the different needs mentioned above, at least if you don't have the programming abilities to write your own custom programs using e.g., Fortran or C.

Recently, I came across Python and found it to be a very powerful tool. In this article, I would like to share my experience and illustrate that even with basic (or less) programming skills it is still possible to create some very useful applications for data analysis and visualization using this language. The article is centered around a few illustrative examples and covers the visualization part — data analysis will be covered in a future article.

Python: a brief review

Python was originally created by Guido van Rossum and is an interpreted programming language (like e.g. Perl) with a clear and easy-to-read syntax. You can write stand-alone applications with Python, but one of it's strengths is its ability to act as glue between different kinds of programs.

The standard introduction to any programming language is the Hello world! program. In Python this is generated by first opening the Python interpreter by typing python on the command line. Your screen should look something like this:

Python 2.3.4 (#1, Jan 21 2005, 11:24:24)
[GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Then the following code is typed:
print "Hello world!"

Python code can also be stored in a file e.g. named script.py. By convention, files containing python code have a *.py extension. The script can be executed by typing python script.py on the command line. The program output will then be written to stdout and appear on the screen. If the following line is added to the top of the file:

#! /usr/bin/python

(assuming that the python executable or a symlink to it exists) and giving the file executable mode with chmod u+x script.py the script can be executed by typing ./script.py on the command line.

Python comes with many modules, either built-in or available for separate download and installation. In this article we will use SciPy, which is a very powerful library of modules for data visualization, manipulation and analysis. It adds functionality to Python making it comparable to e.g. octave and matlab.

Plotting 2-D data

Example 1: Plotting x,y data

The first example illustrates plotting a 2-D dataset. The data to be plotted is included in the file tgdata.dat and represents weight loss (in wt. %) as a function of time. The plotting routine is in the file tgdata.py and the python code is listed below. Line numbers have been added for readability.

     1  from scipy import *
     3  data=io.array_import.read_array('tgdata.dat')
     4  plotfile='tgdata.png'
     6  gplt.plot(data[:,0],data[:,1],'title "Weight vs. time" with points')
     7  gplt.xtitle('Time [h]')
     8  gplt.ytitle('Hydrogen release [wt. %]')
     9  gplt.grid("off")
    10  gplt.output(plotfile,'png medium transparent picsize 600 400')

To run the code, download the tgdata.py.txt file, rename it to tgdata.py, and run it with python tgdata.py. Besides Python, you also need SciPy and gnuplot installed. Gnuplot version 4.0 was used throughout this article. The output of the program is a plot to screen as shown below. The plot is also saved to disk as tgdata.png per line 4 above.

2-D TG data

In line 1, everything from the SciPy module is imported. In order to make use of the various functions of a module, the module needs to be imported by adding an import module-name line to the the python script. In this case it might have been sufficient to import only the gplt package and the io.array_import package. In line 3 the io.array_import package is used to import the data file tgdata.dat into the variable called data as an array with the independent variable stored in column 0 (note that array indices start with 0 as in C unlike Fortran/Octave/Matlab where it starts at 1) and the dependent variable in column 1. In line 4 a variable containing the file name (a string) to which the plot should be stored. In line 6-10 the gplt package is used as an interface to drive gnuplot. Line 6 tells gnuplot to use column 0 as x-values and column 1 as y-values. The notation data[:,0] means: use/print all rows in column 0. On the other hand data[0,:] refers to all columns in the first row.

The gnuplot png option picsize can be a little tricky. The example shown above works when Gnuplot is built with libpng + zlib. If you have Gnuplot built with libgd the required syntax becomes size and the specified width and height should be comma separated.

In order to plot a file with a different file name, we have to open the python source in a text editor and manually change the name of the file to be imported. We also need to change the name of the file copy if we do not want to overwrite the previous plot. This is a little tedious. We can easily add this functionality to our python script by allowing filenames to be passed as command line arguments. The modified script is called tgdata1.py and is shown below.

     1  import sys, glob
     2  from scipy import *
     4  plotfile = 'plot.png'
     6  if len(sys.argv) > 2:
     7          plotfile = sys.argv[2]
     8  if len(sys.argv) > 1:
     9          datafile = sys.argv[1]
    10  else:
    11          print "No data file name given. Please enter"
    12          datafile = raw_input("-> ")
    13  if len(sys.argv) <= 2:
    14          print "No output file specified using default (plot.png)"
    15  if len(glob.glob(datafile))==0:
    16          print "Data file %s not found. Exiting" % datafile
    17          sys.exit()
    19  data=io.array_import.read_array(datafile)
    21  gplt.plot(data[:,0],data[:,1],'title "Weight vs. time" with points')
    22  gplt.xtitle('Time [h]')
    23  gplt.ytitle('Hydrogen release [wt. %]')
    24  gplt.grid("off")
    25  gplt.output(plotfile,'png medium transparent picsize 600 400')

In the first line we have imported two new modules — sys and glob — in order to add the desired flexibility. In Python, sys.argv contains the command line arguments when execution starts. sys.argv[0] contains the filename of the python script executed, sys.argv[1] contains the first command line argument, sys.argv[2] contains the second command line argument and so on. The glob.glob() function behaves as ls in *nix environments in that it supplies filename wildcarding. If no matches are found it returns the empty list (and thus has a len() of zero), otherwise it contains a list of matching filenames. The script can be executed with any desired number of command line arguments. If executed with two arguments e.g. python tgdata1.py tgdata.dat tgdata1.png the first argument is the name of the file containing the data to be plotted, and the second argument is the desired name of the file copy of the plot.

The script works as follows. A default file name for the hard copy of the plot is stored in the variable plotfile (line 4). Then some conditions about the number of given command line arguments are checked. First, if two or more command line arguments are given plotfile is overwritten with argument no. 2 (line 6-7.) Any arguments after the second are silently ignored. For 1 or more arguments given argument 1 is used as the data file name (line 8-9). If no command line arguments are passed, the user is prompted to input the name of the data file (line 10-12). In case of an invalid file name being used for the data file the script prints an error message and exits.

Example 2: Plotting x,y data with error bars

So far we have shown that the gplt package is easy to interface with gnuplot and very effective. However, for scientific use it is often desirable to represent uncertainties for each data point. Although this is possible in gnuplot the gplt interface lacks this functionality. Instead we use the popen package included in the os module. With popen it is possible connect to the stdin (or stdout) of a program through a pipe.

The code below (also available in xrddata.py.txt, the data file is available as xrddata.dat) more or less shows how example 1 is reproduced using popen instead of the gplt package. The major difference is the fact that with popen it is not necessary to import the data to be plotted into Python - instead it is read directly by gnuplot.

     1  import os
     3  DATAFILE='xrddata.dat'
     4  PLOTFILE='xrddata.png'
     5  LOWER=35
     6  UPPER=36.5
     8  f=os.popen('gnuplot' ,'w')
     9  print >>f, "set xrange [%f:%f]" % (LOWER,UPPER)
    10  print >>f, "set xlabel 'Diffraction angle'; set ylabel 'Counts [a.u.]'"
    11  print >>f, "plot '%s' using 1:2:(sqrt($2)) with errorbars title 'XRPD data' lw 3" % DATAFILE
    12  print >>f, "set terminal png large transparent size 600,400; set out '%s'" % PLOTFILE
    13  print >>f, "pause 2; replot"
    14  f.flush()

The code of example 2 produces the output shown below. The error bar plot is created with plot 'filename' using 1:2:(sqrt($2)) with errorbars because in xrddata.dat the standard deviations are equal to the square root of the y-values. This is a special case and usually errors are given explicitly as a third data column in the data file. Thus, an error bar plot is created with plot 'filename' using 1:2:3 with errorbars.

2-D XRPD data

Plotting 3D-data

Example 3:

Now we look at how 3-D data can be represented by using a combination of Python/gnuplot. In order for gnuplot to represent 3-D data it requires that the data is either given by a mathematical expression or stored in a data file. The data file should have either a matrix-format where z-values are given as a matrix with x and y equal to the row and column number, respectively, corresponding to each z-value or a 3 column format where each row represents a data triple with x, y, and z given by the 1., 2., and 3. column respectively. See the gnuplot manual for further details.

The data to be represented in a 3-D fashion in this example is actually a collection of 2-D data files (like the one shown in example 2). Each data file corresponds to an experiment at a different time (with 150 s intervals between experiments) so we have two independent variables and one dependent variable: x(file number/time), y (diffraction angle),and z(counts) distributed across several files. This makes the 3-D data not suitable for plotting — yet.

The script 3ddata_1.py shown below finds all files with a given extension (*.x_y in this case) in the current working directory and creates a list containing their file names (line 5, FILELIST). In line 6 the number of data rows in each file is determined (SIZEX). This information, including the number of data files, is then used to construct an array (DATAMATRIX) with a size of SIZEX by len(FILELIST). In lines 11-12 we cycle through the data files copying the second column in datafile number y is copied to column number y in DATAMATRIX. The array now holds all z-values. This array is only temporary, suitable for data processing before the actual data file for the 3-D plotting is written.

In line 14-22 the data file is written in the (x,y,z) format with x corresponding to time (found by multiplying the file number with the time step), y corresponding to diffraction angle as given by TWOTHETA, z corresponding to counts. In this case we only want to plot the data with diffraction angles between 35-60 (corresponding to data rows 1126-1968). Therefore, only this range is written to file in order to speed up both the process of writing to file and the plotting. In line 24-29 gnuplot is fed input using the popen package.

     1  import os, glob
     2  from scipy import *
     4  EXT='*.x_y'
     5  FILELIST=glob.glob(EXT)
     6  SIZEX = len(io.array_import.read_array(FILELIST[0]))
     7  DATAMATRIX = zeros((SIZEX,len(FILELIST)), Float)
     8  TWOTHETA=io.array_import.read_array(FILELIST[0])[:,0]
     9  TIMESTEP=150
    11  for y in range(len(FILELIST)):
    12                  DATAMATRIX[:,y]=sqrt(io.array_import.read_array(FILELIST[y])[:,1])
    14  file = open("3ddata.dat", "w")
    16  for y in range(len(FILELIST)):
    17          for x in range(1126,1968):
    18                  file.write(repr(TIMESTEP*y)+" "\
    19                  +repr(TWOTHETA[x])+" "+repr(DATAMATRIX[x,y]))
    20                  file.write("\n")
    21          file.write("\n")
    22  file.close()
    24  f=os.popen('gnuplot' ,'w')
    25  print >>f, "set ticslevel 0.0 ;set xlabel 'Time [s]'; set ylabel 'Diffraction angle'"
    26  print >>f, "set pm3d; unset surface; set view 60,75; splot '3ddata.dat' notitle"
    27  print >>f, "set terminal png large transparent size 600,400; set out '3ddata_1.png'"
    28  print >>f, "replot"
    29  f.flush()

If you wanted to write a 3-D data file in the matrix-format, lines 14 through 22 can be replaced with the following code.

file = open("3ddata_matrix.dat", "w")
for x in range(SIZEX):
        for y in range(len(FILELIST)):
                file.write(repr(DATAMATRIX[x,y])+" ")

The 3-D plot produced by the above script is shown below.

3-D (pseudo) XRPD data

The plot above shows that it can be difficult to produce a 3-D plot of non-monotonic data that shows all of the details of the data — some of the smaller peaks are hidden behind larger peaks. It is also difficult to see changes in peak positions as a function of time. In order to bring out these details it is sometimes better to create a 2-D contour plot by projecting the z-values down into the x,y plane. This is achieved by replacing lines 24-29 of 3ddata_1.py with the code below (3ddata_2.py).

f=os.popen('gnuplot' ,'w')
print >>f, "set pm3d map; set palette rgbformulae 30,31,32; set xrange[0:4500]"
print >>f, "set xlabel 'Time [s]'; set ylabel 'Diffraction angle'"
print >>f, "splot '3ddata.dat' notitle"
print >>f, "set terminal png large transparent size 600,500; set out '3ddata.png'"
print >>f, "replot"

The contour plot is shown below.

3-D (pseudo) XRPD data

The 3-D example plots were made with 39 data files, each containing 4096 data rows. The data files are available at this offsite link, and can be unpacked with tar xvfz 3ddata.tar.gz.


In this article a few examples have been given in order to illustrate that Python is indeed a powerful tool for visualization of scientific data by combining the plotting power of gnuplot with the power of a real programming language. It should be noted that all the examples given here could probably have been solved with a combination of e.g. bash, gawk and gnuplot. It appears to me that Python is much simpler and the resulting scripts are more transparent and easy to read and maintain. If heavy data processing is required the bash/gawk/gnuplot might also need the added functionality of e.g. octave. With Python this functionality is in SciPy.

Suggestions for further reading

Manuals, Tutorials, Books etc:

  1. Guido van Rossum, Python tutorial, http://docs.python.org/tut/tut.html
  2. Guido van Rossum, Python library reference, http://docs.python.org/lib/lib.html
  3. Mark Pilgrim, Dive into Python, http://diveintopython.org/toc/index.html
  4. Thomas Williams & Colin Kelley, Gnuplot - An Interactive Plotting Program, http://www.gnuplot.info/docs/gnuplot.html
  5. Travis E. Oliphant, SciPy tutorial, http://www.scipy.org/documentation/tutorial.pdf
  6. David Ascher, Paul F. Dubois, Konrad Hinsen, Jim Hugunin and Travis Oliphant, Numerical Python, http://numeric.scipy.org/numpydoc/numdoc.htm

See also previous articles about Python published in LG.

Other graphical libraries for Python

Gnuplot version 4.0 has been used in the examples throughout this article. Python also works with other plotting engines/libraries:

  1. Grace, see http://www.idyll.org/~n8gray/code/
  2. PGPLOT, see http://efault.net/npat/hacks/ppgplot/
  3. PLplot, see http://www.plplot.org/
  4. matplotlib, see http://matplotlib.sourceforge.net/ (library specifically made for Python)


Anders has been using Linux for about 6 years. He started out with RH 6.2, moved on to RH 7.0, 7.1, 8.0, Knoppix, has been experimenting a little with Mandrake, Slackware, and FreeBSD, and is now running Gentoo on his workstation (no dual boot :-) at work and Debian Sarge on his laptop at home. Anders has (a little) programming experience in C, Pascal, Bash, HTML, LaTeX, Python, and Matlab/Octave.

Anders has a Masters degree in Chemical Engineering and is currently employed as a Ph.D. student at the Materials Research Department, Risö National Laborary in Denmark. Anders is also the webmaster of Hydrogen storage at Risö.

Copyright © 2005, Anders Andreasen. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

A Tale of Three Conferences

By Howard Dyckoff

-- Software Development West 2005, Santa Clara Convention Center
-- Open Source Business Conference (OSBC), Westin St. Francis, San Francisco
-- MySQL User Conference, Santa Clara Convention Center

Three significant conferences in March and April provided evidence of a paradigm shift in our industry. Two were for geeks and one was also for IT managers and Gucci-shoed VCs.

The paradigm shift is in group development methodologies - and it's getting support partly from the increasing acceptance of Open Source Software(OSSw). Because OSSw calls for the collaboration of many individual contributors in far-flung locations, Test Driven Development (TDD) is a rising star in the developer world.

The gurus of software design and technique are now extolling TDD, and are finding that it dovetails nicely with other schools of developer best practice. Many gurus, some grudgingly, now extol TDD because the disciplines they have advocated over the years 'just fall out' of the practice of coding test cases first or simultaneously with each module. Tools like JUnit - freely available and open-sourced - reinforce this shift.

We all know we should do our Design work up-front, and that we should code in small modules to encourage reuse, and that Agile and XP programming methods advocate frequent, small steps, and we should regularly refactor code and test performance during development, and... and.... It is interesting that TDD enforces all of these and other best practices without having to attend a lot of classes and reading at least half a dozen books. TDD Just Works™.

First, there should always be a test-case for each bit of code. Second, writing test cases encourages up-front design. Third, you always have working code at the end of a unit. And fourth, consistent with OSSw strategies, major bugs are uncovered early and dealt with in the development phase. Just like in the case of XP, you can have stylistic variations with TDD - but there is a lot more agreement than disagreement.

The new wrinkle with OSSw is that new applications - and new companies - drive value up the stack. A company like Google doesn't spend all of its time just tweaking its web servers; just take a look at Google Maps with its satellite images. It relies on standard hardware and software to be the base for totally new applications. And that standard software is increasingly become dependent on Open Source.

In fact, it's not just about the software itself: it's the utility of that software to a user or company. Google and Yahoo and eBay provide services to users, and the value of those services drives users to their web sites. But it's not the underlying hardware or the OS that necessarily enhance these services; instead, it's how developers leverage the underlying platforms in new and creative ways.

TDD and OSSw have an overlap, a sweet spot where developers become enabled by tools and methodologies that free them up to do the next thing - which will, of course, be totally awesome and amazing, or so goes the promise of this Brave New World. In either case, it's nice that Linux and its OSSw cousins will be part of that.

Conspiracy theorists will speculate about coincidence of the Santa Clara Convention Center hotel, originally a DoubleTree, becoming a Westin. Does that mean that Westin is the chain of choice for techno-geek? Or an open source Mecca? I think not.

At OSBC, the stars were visionaries, suits, and venture capitalists. Small companies were looking for financing, large companies were testing the Open Source waters, and the model of free software and paid support was being refined and extended into an ecosphere of interrelated companies and development communities.

At the MySQL User Conference [I may use MUC as an acronym occasional], however, the emphasis was different: the stars here were Open Source developers, many now employees of a confident, growing MySQL AB, now 10 years old. Since these folks came from over 40 countries, it was a very festive time.

This year, the MySQL User Conference was managed by O'Reilly Media. This kept things interesting and edgy - and attendees were also encouraged to come to O'Reilly's Web 2.0 conference in SF during the fall.

Technical sessions were only 50 minutes long with a 30 min or longer breaks after every two sessions, and even the Monday tutorials were only 3 hours long. Sessions moved quickly, often with little time for Q&A. By contrast, sessions at SD West were almost double at 90 minutes, and some security workshops spanned two sessions in the same room. I preferred the longer ones, since that allowed attendees to move if the material or presentation wasn't what was expected.

The food at MUC was hotel banquet style, and O'Reilly put a book into each of the conference tote bags [for early registrants, at least.] This contrasts with no books and sandwich boxes at OSBC and SD West. The difference may be that O'Reilly and MySQL AB treated the participants as customers, or even potential authors, and cultivated them more.

Another difference was the approach to internet access. At SD West, there was a limited number of internet-connected machines for emailing - but at OSBC and MUC there were none. The assumption there seemed to be that almost everyone has a personal laptop or pocket PC, and the conference organizers only needed to provide wireless access. We'll probably see more of this as a way to cut conference costs, so be prepared to travel to conferences with a 'device' of your own.

Each of the conferences had their stalwarts who come year after year [that was less so at OSBC, since this was only the second year]. However, informal conversations showed a very high level of satisfaction with the MySQL User Conference. This may be partly due to the tighter subject focus and the clear technical levels of the sessions. It may also have been due to the friendly user community that has emerged around MySQL.

Read on to find out more about the individual conferences and their highlights.

Software Development West 2005, Santa Clara Convention Center

Open Source Business Conference (OSBC), Westin St. Francis, San Francisco

MySQL User Conference, Santa Clara Convention Center

[BIO] Howard Dyckoff is a long term IT professional with primary experience at Fortune 100 and 200 firms. Before his IT career, he worked for Aviation Week and Space Technology magazine and before that used to edit SkyCom, a newsletter for astronomers and rocketeers. He hails from the Republic of Brooklyn [and Polytechnic Institute] and now, after several trips to Himalayan mountain tops, resides in the SF Bay Area with a large book collection and several pet rocks.

Copyright © 2005, Howard Dyckoff. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

Secure Knoppix on CD-ROM for Disaster Recovery

By Edgar Howell


A friend recently sent me a CD with a version of Knoppix designed to enable surfing the Internet without risk. For its intended purpose alone worth looking at, to me the real mind-blower is its use to recover from disaster.

As chance would have it, I had barely started to play around with it a bit when our nephew called: Blue Screen of Death. No back-up. Cool. Yeah, OK, come on over tomorrow afternoon and I'll see what I can do. Windows?! Well...

This thing was put together by people who really know what they are doing. Their use of symbolic links was sheer genius. Well, OK, if you insist, pretty straight-forward — that's what they're there for.

What is Knoppix?

Knoppix is a self-contained mini-version of Linux on CD-ROM. I had heard of it, of course, but never had the time to investigate it. Big mistake. It is well worth being familiar with.

This particular version is intended basically to turn your PC into something like a diskless workstation. You boot from the CD-ROM and it sets up an environment analogous to a chroot-jail without access to any hard drive. In this environment you can surf with Firefox. Even save settings — to diskette or USB. And when you are all done, whatever you haven't yourself explicitly saved somewhere is history.

No virus. No trojan. No spyware. No cookies. Nada. Niente. Rien. Nichevo.

It is worth noting that this was introduced to me by a friend who had no prior experience with GNU/Linux because the software he requires professionally is only available under Windows. But he is using it now for Internet access. Pretty easy to understand. I like Firefox, too.

Putting it to work

OK, so I admit, I haven't gone surfing with it just yet. But is it ever a powerful recovery tool! I like Tom's Boot Disk, which is on the Ultimate Boot CD along with quite a few other useful tools. But you have to experience what this thing can do.

Perhaps it should be pointed out that this is based on Debian and the kernel is a bit old, 2.4.29. But so what? There aren't too many exploits possible if there is nowhere to store anything. And nothing unusual to look at.

Once you boot, you are given a gui — user, not root — under X11 and options to do things needed to surf. That's it. No hard drive. Everything you need to access the Internet and not a single thing more.

However — this is Linux after all — there are the other virtual terminals. All already logged in as root.

Very quickly I was able to put together a simple script with which to establish a network connection. And mounting a drive or partition is no biggey — Knoppix has /etc/fstab all set up for us, mount-points for every single formatted partition.

During the time I needed to use it, the only things I really missed were Midnight Commander and netcat. Apparently netcat is entirely self-contained because there was no no trouble running it copied from a floppy, copied there from SuSE 8.0. On the fly (and without SSH) over the network "cp -R" along with netcat had to substitute for "mc".

Knoppix seems to deal fairly well with "mature" hardware. While X11 baled out on the Pentium 166 with 32 MB, I didn't want to surf on it anyway. And several command-line interfaces as root anywhere is nirvana!

I ought to mention that I was unable to boot the 5-year-old Toshiba notebook (AMD K-5) from the CD. There are numerous options one can enter at boot but none of them helped. My guess is that there is something wrong with a file needed only for the notebook — repeated read errors on one specific block. Proprietary that they are, notebooks are notoriously difficult to configure and deal with anyhow, so no real surprise and not particularly important to me.

Tinkering Under the Hood

Ignoring its original intent, this is a wonderful tool for disaster recovery with both network and mountable devices on which to rescue data. You likely will still want whatever tools you have collected over time to diagnose things like hardware problems. And we're root, so we still do need to be extremely careful about what we do. I had no trouble using fdisk to re-format our nephew's second hard drive and allocate partitions.

The way the people who put this together went about it was very impressive. Instead of hard-wiring everything in, they made skillful use of soft-links. Once I noticed that /etc/hosts etc. were soft-links, it didn't take long to do a script on a floppy to copy what I really wanted from it to /tmp, remove the links and replace them with references to the files in /tmp.

All that was necessary to set up networking was to copy /etc/hosts, /etc/hosts.allow and /etc/hosts.deny from one machine to diskette and then shut it down while our nephew was here and his machine was attached to the network. No long-term approach, but effective in haste.

I looked at the contents of /bin, /sbin, /usr/bin, and /usr/sbin and it would seem that this is a fairly complete Linux distribution: a couple of shells, lilo, miscellaneous mkfs*, awk, sed, ipchains, iptables and so forth. Almost nothing in the way of daemons, window managers or bells-and-whistles, as if anyone should care in the intended environment. Since this CD isn't even half full, you might want to check for your favorite programs before producing your own.

The version I was given was in German but included texts for English prompts at boot. It looks like it ought to be pretty straight-forward to rename two files in order to change to English before burning a CD. And they might serve as patterns for other languages. Pure speculation.

[The ISO image for Knoppix version 3.8.1-2005-04-08 (the latest as of 05 May 2005) is almost 690 MB - not much space for additions! See the mirrors page to download the latest version in several languages. — dsrich]

So our nephew brought his PC over and we connected it to the LAN. Between that and a USB-stick we were able to recover about 90 percent of the stuff he hadn't backed up properly from a drive that seems to have developed a heat-allergy leading to read errors — this drive also had operating system. When all was said and done, he was pretty impressed with what I was able to do.

But was I ever impressed with Knoppix!


If you decide to use this software for Internet access, do be aware of the significance of those root sessions — without password! The only services available are printer and monitor. /etc/hosts and friends are tightly locked down. But it was extremely easy for me to open up network access.

So if some SOB somehow can get some sort of attachment past you and executed... Certainly an unlikely eventuality given the target — not the universe of PCs, not even GNU/Linux, just a variety of a specialized version of Linux. But maybe root should have a password before you burn your own copy.

[A book that discusses this and other Knoppix uses is Knoppix Hacks by Kyle Rankin — dsrich]

[BIO] Edgar is a consultant in the Cologne/Bonn area in Germany. His day job involves helping a customer with payroll, maintaining ancient IBM Assembler programs, some occasional COBOL, and otherwise using QMF, PL/1 and DB/2 under MVS.

(Note: mail that does not contain "linuxgazette" in the subject will be rejected.)

Copyright © 2005, Edgar Howell. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

Shelling your Linux box with Festival

By Maxin B. John

Do you want a bit more "humanized" version of your computer? If you are ready for this adventure, all that you need is a computer with Linux operating system and a little bit of time to download and install the required programs. Now let us have some fun with bash and Festival TTS (Text To Speech) software.

My setup

I am using a Pentium 4 (1.6 GHz) machine with 128 MB of RAM and Redhat Linux 9 installed. The system has an onboard soundcard, which is quite sufficient for this venture. If you want some privacy with your system, I would also recommend a pair of headphones.

Software installation

The software that we are going to use is the Festival speech synthesis system — a Text to Speech program developed by the University of Edinburgh. The power of shell scripting can harness the full potential of the Festival program. Festival comes bundled with Redhat 9 and Fedora distributions. If you don't have it installed, you can download it from http://www.cstr.ed.ac.uk/downloads/festival/1.95/ or install from its rpm package using the usual rpm command.

Let's begin

Just like every C language book begins, let us start with a "Hello World" program. For this type

echo  "Hello World"| festival --tts 

at the command prompt. If everything is working, we should hear "Hello World" through the speakers.

The "Salute" Program

Now let's write a script to play something like "Good morning Sir" whenever we log on to the system. The sound output will dynamically change based on the login time. The shell script goes like this:


dat=`date +%H`

if [ $dat -le $morning ] 
        echo "Good morning sir" |festival --tts

elif [ $dat -gt $morning ] && [ $dat -le $afternoon ]
        echo "Good afternoon sir"|festival --tts

elif [ $dat -gt $afternoon ] && [ $dat -le $evening ]
        echo "Good evening sir"|festival --tts

Save this file as salute.sh in your home directory, then edit .bash_profile in your home directory and add . ~/salute.sh to it. Now whenever you login to your system, you will hear the "sweet salute."

Create a "Virtual Personal Assistant" with cron

Festival can be used along with the cron to create a personal assistant who reminds you about your appointments. Cron is used to schedule programs in Linux. If it is not already running, start the cron deamon by running service crond start as root. Now create your sound alerts — put the following lines into tea.sh in your home directory:

echo  "Don't forget to drink tea at 6.30 "| festival --tts

Now schedule this as a job in the cron with specified date and time. To remind you of this every day at 6.30 P.M , create a file and name it test. In it insert the line given below

30 18 * * *  sh ~/tea.sh

and run the command

crontab test

to schedule the event. To check that the job is correctly scheduled, run crontab -l. The system will speak the message at the specified time. You could even program your parents PC to wish them happy birthday and surprise them.

Get really hot news from BBC World website

Curious to know what is happening around the world? You can create a News reader to fetch the hot news from the BBC world website and read it for you. For this purpose, we are going to use the RSS feed of that website.

Really Simple Syndication (RSS) is a technology for sharing information on the web. It simple and well-established XML format used to syndicate headlines. Once a website creates an RSS file they have created a means to allow others to retrieve their headlines. Popular news sites like Yahoo!, BBC News, The New York Times along with tens of thousands of personal blogs and independant media sites all publish RSS feeds that we can use.

This shell script will fetch the RSS feed from the BBC World website and then process its headlines and contents. The output is then piped to the Festival program so we can hear the hottest news happening around the world.

#Shell script to fetch and read the hottest news from BBC world using Festival
#Here we specify the url
#specify the proxyserver here, if you have one
headarg="-10"  # default is five headline, change it to increase the no of feeds
#specify the proxy with -x option in curl program, if you
#don't use one, then remove the '-x "$proxy"' part
curl --silent -x "$proxy" "$url" | grep -E '(title>|description>)' | \
  sed -n '4,$p' | \
  sed -e 's/<title>//' -e 's/<\/title>//' -e 's/<description>/   /' \
      -e 's/<\/description>//' | \
#Here we pump the output to Festival
   head $headarg | fmt|festival --tts
exit 0

Save the code as rss.sh and run it as

sh rss.sh

Make sure that you are connected to internet before running this code, since the curl program needs to contact the bbcworld website and fetch the contents.

The god of small things

Sometimes I wish I could hire somebody to read me books or boring man pages. Nowadays I don't read man pages — I just listen to them! For example, to hear the man page of cron:

man cron | festival --tts  

Curious to know the name of people logged on to your system?

 who | awk " print {$1};"| festival --tts

There are lots of things you can have read to you - give them a try!


The power of Festival combined with shell scripting is huge — the sky is the limit.

Further Reading

I haven't discussed the theory behind the text to speech as it is out of the scope of this article. If you are curious, visit http://fife.speech.cs.cmu.edu/festival/manual-1.4.1/festival_toc.html for further details.

To learn more about Shell scripting, check out:

Linux Shell Scripting with Bash by Ken O. Burtch
Wicked Cool Shell Scripts by Dave Taylor

[BIO] Maxin B. John works for HCL Infosystems Ltd, Pondicherry and is an MCA graduate from Govt. Engg. College, Thrissur. He likes to experiment with Python and Bash. He thanks his guru Mr. Pramode C.E for introducing him to the wonderful world of Linux.

Copyright © 2005, Maxin B. John. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

Right To Your Own Devices

By Kapil Hari Paranjape


Linux 2.4.x had the Logical Volume Manager (LVM) and other multi-disk/multi-partition block device constructs. These have been enhanced by the Device Mapper in Linux 2.6.x. Here is a one line summary:

You can choose any sequence of blocks on a sequence of block devices and create a new block device some of whose blocks are identified with the blocks you chose earlier.
That'll take a while to chew on. Meanwhile here are some ways you can use the device mapper: There is a catch, of course. To do all this with the root device, some changes need to be made at boot time. Rather than get diverted, we will concentrate on learning how to use the device mapper - to do this we will use “dummy” loop devices instead of “real” disks. After you have gained some confidence, you can move on to real disks and perhaps even the root device.

How To Fake It With A Loop

In Unix, everything is a file. Even a block device like /dev/hda2 which is meant to be read in “chunks” called blocks, can be read byte-by-byte like a file. The loop device allows us to reverse this asymmetry and treat any file like a block device. Activate loop devices for your Linux with modprobe loop (as root) if necessary.

To demonstrate this without risking serious damage to useful files, we will only use empty files. First of all, create an empty file like so:

    dd if=/dev/zero of=/tmp/store1 bs=1024 seek=2047 count=1
This creates a file full of nothing and 2 Megabytes in size. Now we make it into a block device:
    losetup /dev/loop1 /tmp/store1
We then operate with this block device just as we would with any other block device:
  1. Check its size in 512-byte blocks
            blockdev --getsize /dev/loop1
  2. Make a file system on it
            mke2fs /dev/loop1
  3. Mount this file system somewhere
            mount /dev/loop1 /mnt
After this you can use /mnt just as you would any other file-system - the changes will be written to /tmp/store1. When you get tired of playing with the loop blocks, you put them away with commands like losetup -d /dev/loop1.

We will use loop devices like /dev/loop1, /dev/loop2 and so on as the building block devices in what follows.

Step Into My Parlor...

...said the device mapper to the block device. If it is not already activated, load the device mapper for your Linux with modprobe dm-mod (as root.) The device mapper can take any block device under its wing with a command like

    echo 0 $(blockdev --getsize /dev/loop1) linear /dev/loop1 0 | \
        dmsetup create new
This creates a “new” block device /dev/mapper/new; but this is not really new data. Reading from this block device returns exactly the same result as reading directly from /dev/loop1; similarly with writing to this block device. Looks a lot like the same old blah in a new block device! So you could get rid of this block device by dmsetup remove new.

Of course, you can do things differently. For example, you can take only half of /dev/loop1 as your block device:

    SIZE1=$(blockdev --getsize /dev/loop1)
    echo 0 $(($SIZE1 / 2)) linear /dev/loop1 0 | \
        dmsetup create half
The remaining half (which could be the bigger “half” if /dev/loop1 is odd-sized!) is then also available for use. You could use it in combination with /dev/loop2 to create another block device:
    REST1=$((SIZE1 - $SIZE1 / 2))
    echo 0 $REST1 linear /dev/loop1 $((SIZE1 / 2))  > /tmp/table1
    echo $REST1 $(blockdev --getsize /dev/loop2) \
        linear /dev/loop2 0 >> /tmp/table1
    dmsetup create onenahalf /tmp/table1
Let us try to understand this example and what each of the three numbers on each line of /tmp/table mean. The first number is the starting sector of the map described, the second number is the number of sectors in the map. The word linear is followed by the name of the original device that the map refers to; this is followed by the sector number of the first sector (of this original device) which is assigned by this map. Read that again!

So you can slice and splice your disks as you like - but there is a small cost, of course. All operations to these new block devices go through the device mapper rather than directly to the underlying hardware. With efficient table management in the kernel, this overhead should not slow down things perceptibly.

Notice how I slipped in (clever me!) the use of “tables” that contain the mapped device descriptions. If you are planning to use mapped devices a lot and don't want to forget your settings, such tables are the way to go. Don't worry - you can always get the table of any device like /dev/mapper/new by

    dmsetup table new
In the output, the original block device will appear as major:minor, so you will have to figure out what the device is actually called if you need the table in human readable form. (Hint: Try
    ls -l /dev | grep "$major, *$minor"
or something very like it.) Don't forget to run
    dmsetup remove half
    dmsetup remove onenahalf
when you are through.

Perhaps you are one of those people who own multiple disks configured so that reading n bytes from one of them is slower than reading n/2 bytes from two of them; this may happen because your disk controller is capable of multi-disk operations in parallel or because you have multiple disk controllers. The device mapper can help you to speed up your operations.

    SIZE=$(( $(blockdev --getsize /dev/loop1) + \
         $(blockdev --getsize /dev/loop2) ))
    echo 0 $SIZE striped 2 16 /dev/loop1 0 /dev/loop2 0 | \
        dmsetup create tiger
Now reads/writes from /dev/mapper/tiger will alternate (in 16 sector chunks) between the two devices; you will also have combined the disks into one as in the linear case.

Snapshots and COWs

There may be a number of reasons why you may want to stop all writes to your block device but not want the system to come to a grinding halt.

Regular backups
“Classically” machines were put in single-user mode to take backups. Backing-up a “live” system has the risk of incomplete or corrupted files and erroneous time stamps.
This jazzy new screen saver you want to install - what changes is it going to make to your file-system? You want to find out.
...or other read-only physical devices. Say you want to set up your system on a CDROM but still want to allow local “ephemeral” changes that are discarded when the system reboots.
The solution is re-direction. Effectively you tell the processes, “Look behind you!” and in-a-snap put a layer between the process and the device. Activate the snapshot feature of the device mapper with modprobe dm-snapshot if necessary.

Let us start then with a device which is managed by the device mapper. For example it could be created by

    SIZE1=$(blockdev --getsize /dev/loop1)
    SIZE2=$(blockdev --getsize /dev/loop2)
    cat > /tmp/table2 <<EOF
    0 $SIZE1 linear /dev/loop1 0
    $SIZE1 $SIZE2 linear /dev/loop2 0
    dmsetup create base /tmp/table2
Now assume that you have put a file system on this device with a command like mke2fs /dev/mapper/base; and suppose you have begun using this file system at /mnt with the command mount /dev/mapper/base /mnt.

We will now take a “snapshot” of this file-system - in slow motion! The following steps have to be run quite quickly (say with a script) on a running system where this file-system is being changed actively.

First of all you create a duplicate of this device. This is not just for safety - we will be changing the meaning of /dev/mapper/base without telling the file-system!

    dmsetup table base | dmsetup create basedup
Next we prepare our COW (copy-on-write) block device by making sure the first 8 (or whatever you decide is your chunk size) sectors are zeroed.
    dd if=/dev/zero of=/dev/loop3 bs=512 count=$CHUNK
Now we suspend all I/O (reads/writes) to the base device. This is the critical step for a running system. The kernel will have to put to sleep all processes that attempt to read from or write to this device; so we want to be sure we can resume soon.
    dmsetup suspend base && TIME=$(date)
The next step is to use the COW to clone the device:
    echo 0 $(blockdev --getsize /dev/mapper/basedup) \
        snapshot /dev/mapper/basedup /dev/loop3 p 8 | \
        dmsetup create top
What this says is that from now on reading from /dev/mapper/top will return the data from /dev/mapper/basedup unless you write “on top” of the original data. Writes to top will actually be written on /dev/loop3 in chunks of size 8 sectors. If you have used multiple transparent plastic sheets one on top of the other (or “Layers” in GIMP) the effect is similar - what is written on top obscures what is below but wherever nothing is written on top you see clearly what is written on the lower layer.

In particular, we can now make sure that all changes to the underlying block devices are “volatile.” If we execute the following commands (we'll bookmark this as 'Point A' for later use) -

    dmsetup table top | dmsetup load base
    dmsetup resume base
we will have replaced the file-system under /mnt with another one where all changes actually go to /dev/loop3. When we dismantle this setup, /dev/loop1 and /dev/loop2 will be in exactly the state that they were in at time $TIME.

If /dev/loop1 and /dev/loop2 are on non-writable physical media (such as a CDROM), whereas /dev/loop3 is on a writable one (such as RAM or hard disk), then we have created a writable file-system out of a read-only one!

This solves the last problem in our list above - but what about the first two? To tackle the second problem we must have some way of comparing the new file-system with the older one. If you try to mount /dev/mapper/basedup somewhere in order to this, you will find that Linux (the kernel!) refuses to let you do this. Instead we can create yet another device:

    echo 0 $(blockdev --getsize /dev/mapper/basedup) \
        snapshot-origin /dev/mapper/basedup | \
        dmsetup create origin
You can now mount /dev/mapper/origin somewhere (say /tmp/orig) and compare the original file system with the current one with a command like
    diff -qur /tmp/orig /mnt
What happens if you write to /tmp/orig? Check it out and you'll be mystified for a moment.

The analogy of plastic sheets breaks down here! All writes to /tmp/orig go directly to the underlying device basedup but are negated on /dev/loop3 so as to become invisible to reads from /mnt. Similarly, reads from /tmp/orig ignore whatever changes were made by writing to /mnt. In other words the original file system has been forked (and orthogonally at that!) and /dev/loop3 actually stores both negative and positive data in order to achieve this. No plastic sheet can be made to work like this!

To see why this is useful, let us see how it solves the problem of backups. What we want is to get a “snapshot” view of the file-system but we want to continue using the original system. So in this case we should not run the commands at point A above. Instead we run the commands here, at point B:

    dmsetup table origin | dmsetup load base
    dmsetup resume base
Now all writes to /mnt will go onto the original device, but these changes are negated on /dev/mapper/top. So if we mount the latter device at (say) /tmp/snap, then we can read a snapshot of the files at time $TIME from this directory. A command like
    cd /tmp/snap
    find . -xdev | cpio -o -H new > "backup-at-$TIME"
will provide a snapshot backup of the file-system at time $TIME.

We could also have taken such a snapshot at Point A with the commands

    cd /tmp/orig
    find . -xdev | cpio -o -H new > "backup-at-$TIME"
The main difference is that the changes to /dev/mapper/top are volatile! There is no way to easily dismantle the structure created under (A) without losing all the changes made. In the backup context you want to retain the changes; at Point B you run
    dmsetup suspend base
    dmsetup remove top
    dmsetup remove origin
    dmsetup table basedup | dmsetup load base
    dmsetup resume base
and you are back to business as usual. If you were to run this at Point A the results would be quite unpredictable! What would be the status of all those open files on /dev/mapper/top? A number of hung processes would be the most likely outcome - even some kernel threads could hang - and then perhaps break!

For Your Eyes Only

Say you have a laptop or CD which carries some valuable data - valuable not just to you but to anyone who has it. (When, Oh! When will I ever get my hands on such data). In this case backups are no good. What you want is to protect this data from theft. Assuming you believe in the strength of current encryption techniques you could protect it by encrypting the relevant file. This approach has some serious problems:

For these and a possible host of other reasons you may want to encrypt the entire block device. The device mapper offers a way to do this. Activate the encryption service of the device mapper with modprobe dm-crypt if necessary. Also activate some encryption and hashing mechanism by commands like modprobe md5 and modprobe aes if necessary.

First of all you need to generate and store your secret key. If you use AES as indicated above then you can use a key of length up to 32 bytes which can be generated by a command like

    dd if=/dev/random bs=16 count=1 | \
      od --width=16 -t x2 | head -1 | \
      cut -f2- -d' ' | tr -d ' ' > /tmp/my_secret_key
Of course, you should probably not output your secret key to such a file - there are safer ways of storing it: If you use the third option you will need to use a passphrase - you must remember this passphrase as well. One way to do that is to use this passphrase often - use it... or lose it!

You can now setup the encrypted device

    echo 0 $(blockdev --getsize /dev/loop1) \
      crypt aes-plain $(cat /tmp/my_secret_key) 0 /dev/loop1 0 | \
      dmsetup create mydata
You can then make a file-system mke2fs /dev/mapper/mydata on this block device and store data on it after mounting it somewhere with mount /dev/mapper/mydata /mnt. All the data written to /mnt will then be transparently encrypted before storing it in /dev/loop1. When you are through you unmount the device and dismantle it as before:
    umount /mnt
    dmsetup remove mydata
The next time you want to use the device you can set it up with the same command as above (providing you supply the secret key in /tmp/my_secret_key). Of course, you shouldn't rune mke2fs on the device a second time unless you want to erase all that valuable data!

Getting To The Root Of The Problem

All the steps given above can be carried out on any block device(s) in place of the loop devices that were used. However, when the block device is the root device then life gets a little more complex. (Roots generally are complex).

First of all we need to put the root device under the control of the device mapper; this is best done with an initial RAM disk (or initrd). Even after this is done, we need to be careful if we are trying to run some of the above commands for the root file system on a “live” system. In particular, it is not advisable to suspend I/O on the root file system without deep introspection! After all this means that all processes that make a read/write call to the root file system will be put to sleep.

Here is one way around the problem. Create a temporary file system

    mount -t tmpfs tmpfs /mnt
To this file system copy all the files that are necessary in order to perform the changes - in particular, you need /sbin/dmsetup, /bin/sh, the /dev files and all shared libraries that these programs depend on. Then you run chroot /mnt. After this you can run a script or (if you type quickly and without errors!) a sequence of commands that will suspend the root device map and make relevant changes to it - for example, to take a snapshot. Don't forget to resume the root device before exiting the chroot.

After word

Given the complexity of the various operations, it is probably best to produce a shell script or even a C program that carries out the tasks. Luckily, the latter has already been implemented - the Linux Logical Volume Manager version 2 does carry out most of the tasks described above quite “automagically.” Setup and use of encryption is greatly simplified by the cryptsetup program. Why then did I write this article?

I originally came upon dmsetup while trying to create a read-only root file system for a “live” CDROM. Unfortunately, the LVM2 tools are not useful as they only look at the use of snapshots for backups - clearly they don't care for COWs! The only resource that I found for this was the RedHat Mailing list archives. There are now tools which come with live CD's that make use of dmsetup; for example I came across this link which explains how UBuntu does it.

Of course, using dmsetup allowed me to get as “close to the metal” as is possible without writing real programs...

This document was translated from LATEX by H EVEA.

[BIO] Kapil Hari Paranjape has been a ``hack''-er since his punch-card days. Specifically, this means that he has never written a ``real'' program. He has merely tinkered with programs written by others. After playing with Minix in 1990-91 he thought of writing his first program---a ``genuine'' *nix kernel for the x86 class of machines. Luckily for him a certain L. Torvalds got there first---thereby saving him the trouble (once again) of actually writing code. In eternal gratitude he has spent a lot of time tinkering with and promoting Linux and GNU since those days---much to the dismay of many around him who think he should concentrate on mathematical research---which is his paying job. The interplay between actual running programs, what can be computed in principle and what can be shown to exist continues to fascinate him.

Copyright © 2005, Kapil Hari Paranjape. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

Lock It Down With Arno's iptables-firewall

By S. Keeling

Executive Summary

This describes my experiences in transitioning from a homebuilt iptables firewall scriptii to "Arno's iptables-firewall" (AIF)v, from Arno van Amersfoort (). AIF can produce a robust iptables based firewall, even when wielded by a relative newcomer to iptables firewalls. The time between reading the documentation and dropping it into place could be as little as an hour.


AIF is described in Arno's README as "Arno's iptables firewall - Single- & multi-homed firewall script with DSL/ADSL support". It is (C) Copyright 2001-2005 by Arno van Amersfoort. It's free software, licenced under the GNU General Public License.

If you decide you want to try using this, I strongly recommend you READ THE FAQ on Arno's website, especially if you join the mailing list in hope of support. Arno points out in there that if you haven't done so, your pleas for assistance will be cheerfully ignored until you do.

Of course, considering this is the Twenty-first Century:


[fscking lawyers (no, that's not a typo)... Sorry for shouting.]

When I started writing this, I was working with version 1.8.2, which is apparently over a year old. At Arno's suggestion, I've since upgraded to the latest release, version 1.8.3-RC3 (April 9, 2005). So, if I've missed anything important in updating this, that's why it may appear confused. Certain example listings were left alone, but I've tried to cover the important bits of the latest version.

Some of Arno's comments on my original article are as follows:

   Thanks for the article. I like it a lot.
   May I suggest to have a look at the latest 1.8.3RC3 (instead of 1.8.2a
   stable, which is over 1 year old)? It has a lot (and I mean A LOT)
   [of] improvements like:
  • 80-character long README & config file (one of your bothers). Furthermore the config file layout is improved a lot.
  • DMZ support (ie. to shield WiFi nets).
  • Multiple external (internet) interfaces.
  • Multiroute NAT (for load balancing internet connections).
  • MAC address filtering.
  • Full (transparent) proxy support.
  • The execute bit isn't set anymore on this version's config file either. ;-)
  • ...and...
  • version 1.8.4-stable, with even more improvements, will be out soon.

Nice guy, that Arno. When I started on this, I (mis)corrected his usage of "DRDOS" to "DDoS" (or "Distributed Denial of Service"xi), and that was the wrong thing to do. He actually means "Distributed Reflection Denial Of Service", something I'd never heard of. I took his advice. For me, just getting config files with eighty character long lines is a terrific improvement.

Many thanks to the helpful and patient staff at LG for all their support in producing this article, and the same to Arno van Amersfoort for having made any of it possible.


My situation is this (which is not to suggest you need to be doing something similar):

Previously, I'd used a handcrafted set of iptables commands which pretty much ruthlessly, and blindly, locked everything out through sheer brute force. Anything the firewall saw that wasn't connected to an ESTABLISHED or RELATED IP process previously initiated by my actions, was simply logged and dropped.

This is generally not difficult to do, but at times, it is. How to let NTP servers' replies back in? I struggled with that for months. Googling for help on that produced so many conflicting examples (some of which worked, some not), it was infuriating. I was always on the lookout for an alternative. I wanted something somewhat like my existing setup, but more robust, more suspicious, more dynamic, and smarter overall about networking than I am.

Over the years, I've tried things others suggested, but they always seemed a bit wrong. I don't want the thing to fire up an X Window GUI forcing me to point and click my way through the configuration. If I already knew everything there was to know about TCP/IP and networking, that might be helpful, but I don't. Besides, that's just not how I do things. I want something with few moving parts, all of which have their own home directories, and which can be expected to stay there. I don't want the thing to fail some morning because somebody changed some obscure library I'd never heard ofvi.

On "Sun Jan 9 12:18:45 MST 2005"i, I found mention of "Arno's iptables-firewall" (AIF). Now that I've finally taken some time to get it working and try it out, I'm happy to say it appears to be just what I was looking for. Arno's system is intended for far more complicated setups than my own (for instance, it supports NAT and VPNs), but it easily scales down to my needs. It thinks of an external (connected to the internet) and an internal (LAN) interface. In fact, the latest release has been updated to handle multiple external interfaces.


It's not difficult. You should be able to get the mechanics of this done in half an hour, depending on how careful you want to be. Budget an hour for the job since there's some reading to do beforehand.

Go to his download page and grab it. It's ~54 kilobytes. It'll be in the form of a tarball (gzipped tar archive). cd to somewhere safe, make a new directory to hold it, and extract it:

   mkdir ~/dwn
   cd ~/dwn
   tar xzf /path/to/arno-iptables-firewall-1.8.3-rc3.tgz

That creates a new directory, ~/dwn/arno-iptables-firewall-1.8.3-RC3:

   total 276
   drwxrwxr-x    2 keeling  keeling      4096 Apr  9 06:20 ./
   drwxr-xr-x   27 keeling  keeling      4096 Apr 18 10:50 ../
   -rw-r--r--    1 keeling  keeling     20063 Apr  9 06:20 CHANGELOG
   -rwxr-xr-x    1 keeling  keeling     13580 Apr  9 06:20 fwfilter*
   -rw-r--r--    1 keeling  keeling     18010 Apr  9 06:20 gpl_license.txt
   -rw-rw-r--    1 keeling  keeling      1467 Apr  9 06:20 iana-reserved-nets.txt
   -rw-rw-r--    1 keeling  keeling     31674 Apr  9 06:20 iptables-firewall.conf
   -rw-r--r--    1 keeling  keeling     29755 Apr  9 06:20 iptables-firewall.conf.example
   -rwxr-xr-x    1 keeling  keeling    112070 Apr  9 06:20 rc.iptables*
   -rw-r--r--    1 keeling  keeling     16887 Apr  9 06:20 README
   -rw-r--r--    1 keeling  keeling      2318 Apr  9 06:20 syslog.conf.Debian
   -rw-r--r--    1 keeling  keeling      1202 Apr  9 06:20 syslog.conf.RedHat

They're all flat ASCII text files. Now, read the README. It starts out by mentioning it's free software, licenced under the GNU General Public License. It goes on to explain what the various files are for. One of the most welcome things to me in the latest version is Arno has chopped the line length in most of his stuff to a much more readable eighty characters or so. He appears to like lines that are about a hundred characters long, and displaying them in an eighty character wide xterm (or console) made them very difficult to read. Mostly, that's no longer a problem. You'll still see remnants of this, in his actual firewall script for example.

Following the file descriptions, he lists "Some IMPORTANT (security) information". Following that is a "Quick setup" section, a section on WHAT TO DO IN THE CONFIG FILE, and some info you may need in rebuilding your kernel if it doesn't already have iptables support (:-O really?!? I haven't seen any distributions supplying pre-compiled kernels without iptables support, or maybe this is in case you're having trouble rolling your own... I'll have to go back and re-read that bit sometime).

The old version flatly refused to work if you were using a 2.2.x kernel and ipchains. The latest version detects 2.2.x and ipchains and continues (I haven't tested this).

The README used to suggest installation instructions you might use, but perhaps he's blended those into the other sections? I notice there's also a bit at the top of the firewall script itself. However, I suggest you use common sense and figure out how you need it done for your system. They're just a couple of scripts and config files. How difficult can it be? :-)

On the other hand...

What I Did - Configuration

As root:

   $ cp rc.iptables /etc/init.d
   $ vi /etc/init.d/rc.iptables

A few lines into the file, you'll see CONFIG_FILE. Change that to say:


Now make it executable (Arno suggests this should be 700; you decide):

   $ chmod 744 /etc/init.d/rc.iptables

If necessary:

   $ chown root:root /etc/init.d/rc.iptables

Now make a home for the config file(s):

   $ mkdir /etc/firewall
   $ chmod 755 /etc/firewall
   $ cp iptables-firewall.conf /etc/firewall
   $ chmod 600 /etc/firewall/iptables-firewall.conf
   $ vi /etc/firewall/iptables-firewall.conf

If you're going to use them, you now want to create iptables-blocked-hosts, iptables-custom-rules, and iptables-mac-addresses (make sure those are the names mentioned, down at the bottom, in the config file). I notice they're commented out in the latest version's config file. These need to be at least one blank line long, so vi them, insert a carriage return, save, and exit.

Now go back to the top of iptables-firewall.conf, inserting the important stuff telling AIF how it should do things for your system. This part gets a little hairy, so you might like to peruse the mailing list archives to see if you can find explanations of how to do this correctly. The FAQ also mentions a few little syntactical nits that it might help to know. I went with Arno's suggestion and changed as little as I could. There are lots of comments explaining what things do and when you might want to use them.

Yes, that is a little vague. Sorry, but this bit is the so-called kernel of what you need to learn about iptables for AIF to work for you the way you want it to work. What is it you want to do, and what is there that you have to work with? DSL/ADSL? Dialup ppp? WiFi from a broadband modem? What ports do you want to open up to the crackers? Do you know what you're doing with iptables, or are you a dilettante like me?

You do want to tell it what your external interface is:

in my case ("ppp+" covers ppp0, ppp1, ppp2, etc.) Then:
takes care of my NIC along with:

One of the things mentioned in the FAQ is you don't want or need "" (the loopback interface) mentioned anywhere. "Ah, so I guess I don't need to do anything about my caching nameserver? Uhh..." Well, the MaraDNS man page has a good section on this. Later.

Also, you might like to take my chmod commands above with a grain of salt. I don't see a great deal of point in locking things down so group and other can't read them. However, that is how Arno suggests they be. You make up your own mind. To me, if a config file doesn't contain any sensitive information, it's a little silly to make it owner:group read-only.

What I Did - Implementation

Once you've solved the config file problem, you then need to figure out how you want it executed. If you want it to come up at boot time, you'll need init script links in /etc/rcN.d directories. Arno mentions how to do this at the top of the firewall script itself. You can do it by hand, but most distributions have some sort of tool you can use to manage those links.

For my purposes, I wanted it controlled by interface initialization, and that fits well with Debian's /etc/ppp/ip-up.d/ system. Any script in that directory will be run when the interface is brought on line and configured. There's also /etc/ppp/ip-down.d/ which works similarly in reverse.

Now, take a look in /etc/network. Just like ppp, there are if-up.d "-ish" (among others) directories in there too.

In my case, I created the script /etc/ppp/ip-up.d/00iptables_firewall. It needs only two lines: the "shebang", and a line to call rc.iptables:

   /etc/init.d/rc.iptables start

Now, "chmod 744 /etc/ppp/ip-up.d/00iptables_firewall". Similarly, you need a script in /etc/ppp/ip-down.d that will take the firewall down when the interface goes down:

   # if [ ! "$(/sbin/ifconfig | /bin/grep eth0)" ]; then
      /etc/init.d/rc.iptables stop
   # fi

Notice the commented out lines. When I started on this, that did a sanity check first: if, by chance, I happen to have both ppp0 and ethN up and connected, it would be a little silly to take down the firewall if only one of the interfaces was being taken down. However, now that I think about it, that might be a little dumb. I'm still re-thinking that bit.

Make sure that file's executable too.

The last thing you might want to do is tweak /etc/syslog.conf to tell syslogd to send firewall related messages to /var/log/firewall. In the latest version, this is optional. Arno was kind enough to include a couple of examples of syslog.conf that you can work with. I just did this: look for the first instance of "kern.*" and change that to "kern.!=debug", then add a new line that points kern.=debug at /var/log/firewall. Now, I have:

   auth,authpriv.*                 /var/log/auth.log
   *.*;auth,authpriv.none          -/var/log/syslog
   #cron.*                         /var/log/cron.log
   daemon.*                        -/var/log/daemon.log
   kern.!=debug                    -/var/log/kern.log
   lpr.*                           -/var/log/lpr.log
   mail.*                          -/var/log/mail.log
   user.*                          -/var/log/user.log
   uucp.*                          /var/log/uucp.log

   # Logging for iptables
   kern.=debug                     -/var/log/firewall

Once you change /etc/syslog.conf, you need to touch the new logfile (only do this if it doesn't already exist):

   $ touch /var/log/firewall

and you need to cycle syslogd to make it re-read its config file:

   $ kill -HUP $(pidof syslogd)

If you don't have "pidof", you'll have to determine syslogd's PID and insert that there instead. "ps fax | grep syslogd" says this:

   (0) keeling /home/keeling_ ps fax | grep syslogd
     242 ?        S      0:00 /sbin/syslogd
   30307 pts/2    S      0:00          |       \_ grep syslogd

so you need to say "kill -HUP 242"

If you've done the syslog.conf fiddle, make sure you've set "FIREWALL_LOG=/var/log/firewall" and "LOGLEVEL=debug" in /etc/firewall/iptables-firewall.conf. Hopefully, you have logrotate on your system, else the log file will just grow and grow, eventually filling your filesystem.

This is "/etc/logrotate.d/firewall":

   /var/log/firewall {
           rotate 7
           create 0640 root adm

Now, you should be good to go. I would suggest the first (few?) time(s), you should run /etc/init.d/rc.iptables by hand ("chmod 644 /etc/ppp/ip-up.d/00iptables_firewall" first, bring up the connection, then run "/etc/init.d/rc.iptables start"). It spits out quite a few messages mentioning what it's doing. You should peruse that output (see the example below), ensuring it's doing what you want. This is also when you'll get a chance to do something about any errors found in the config file. "iptables -nL | less" will show the firewall rules. "tail -f /var/log/firewall" will show you the effect of those rules.

If you can't figure out how to make AIF do something you did before, Arno has thought about that and made exceptions possible. Take the iptables rule you used before and stuff it into iptables- custom-rules. Of course, be warned that this may compromise whatever AIF is attempting to do for you. However, if you were doing it before, you're probably no worse off now than you were.

In my case, when I was using the older version, I had this in my /etc/firewall/iptables-custom-rules to make chrony/NTP work:

   # man iptables says using a name that must be resolved via remote
   # DNS is a Really Bad Idea.  Sigh.
   iptables -t filter -I INPUT -s 0.pool.ntp.org -m tcp -p tcp --dport 123 -j ACCEPT
   iptables -t filter -I INPUT -s 1.pool.ntp.org -m tcp -p tcp --dport 123 -j ACCEPT
   iptables -t filter -I INPUT -s 2.pool.ntp.org -m tcp -p tcp --dport 123 -j ACCEPT

With the new version, however, I've commented those out and used "HOST_OPEN_TCP" in the config file instead:

   HOST_OPEN_TCP="0.pool.ntp.org>123 1.pool.ntp.org>123 2.pool.ntp.org>123"

AIF In Action

Take a look at what it's doing. Here's an example of rc.iptables output:

   Arno's IPTABLES Firewall Script v1.8.3-RC3
   Sanity checks passed...OK
   Detected IPTABLES module... Loading additional IPTABLES modules:
   All IPTABLES modules loaded!
   Setting default secure policies.
   External (internet) interface(s) (EXT_IF)   : ppp+
   Configuring /proc/.... settings
   Enabling anti-spoof with rp_filter.
   Disabling the logging of martians.
   Disabling the acception of ICMP-redirect messages.
   Setting the max. amount of simultaneous connections to 4096 (default).
   Enabling reduction of the DoS'ing ability.
   Disabling ECN (Explicit Congestion Notification).
   Using loglevel debug for syslogd.
   Flushing rules in the filter table.

   Setting up firewall rules
   Accepting packets from the local loopback device.
   Enabling setting the maximum packet size via MSS.
   Enabling mangling TOS.
   Logging of INVALID packets enabled.
   Reading custom IPTABLES rules from /etc/firewall/iptables-custom-rules
   Logging of ICMP flooding enabled.
   Setting up INPUT policy for internal interface(s) eth0
   Logging of stealth scans (nmap probes etc.) enabled.
   Logging of packets with bad TCP-flags enabled.
   Logging of fragmented packets enabled.
   Logging of access from reserved addresses enabled.
   Setting up anti-spoof rules.
   Logging of DHCP broadcasts disabled.
   Logging of probable "lost connections" disabled.
   Logging of explicitly blocked hosts enabled.
   Logging of denied local output connections enabled.
   Logging of denied LAN (forward) output connections enabled.
   Packets will NOT be checked for private source addresses.
   Denying the whole world to send ICMP-requests(ping).
   Logging of dropped ICMP-request(ping) packets enabled.
   Logging of dropped other ICMP packets enabled.
   Logging of possible stealth scans enabled.
   Logging of (other) connection attempts to PRIVILEGED TCP ports enabled.
   Logging of (other) connection attempts to PRIVILEGED UDP ports enabled.
   Logging of (other) connection attempts to UNPRIVILEGED TCP ports enabled.
   Logging of (other) connection attempts to UNPRIVILEGED UDP ports enabled.
   Logging of other IP protocols (non TCP/UDP/ICMP) connection attempts enabled.
   Setting up FORWARD policy for internal interface(s) eth0:
    Allowing all TCP ports
    Allowing all UDP ports
    Allowing all IP protocols
   Security is ENFORCED for the external interface(s) in the FORWARD chain.
   (Re)loading list of BLOCKED hosts (blackhole) from /etc/firewall/iptables-blocked-hosts

   Apr 18 14:47:12 All firewall rules applied.

Woof! You should see the "iptables -nL" output! Some of this stuff, I'd only vaguely heard of. Take a gander into the rc.iptables script itself to see all that it's doing. First, the "Sanity checks", then it modprobes all the necessary kernel modules (again, some of which I'd never heard of). Arno's doing all the obscure:

  echo ${some_integer} > /proc/sys/net/blah/blah

that I've never managed to either take the time to understand or find a decent reference for. Finally, he defines various firewall rulesets.

One of the neat things about this is, instead of the fairly bland:

   Mar 16 07:24:38 localhost kernel: IN=ppp0 OUT= MAC= \
      SRC=xxx.xxx.xx.xx DST=xxx.xxx.xxx.xx LEN=48 TOS=0x00 \
      PREC=0x00 TTL=102 ID=59091 DF PROTO=TCP SPT=3946 DPT=6348 \
      WINDOW=64240 RES=0x00 SYN URGP=0

I now see things like this:

   Apr  6 12:41:06 localhost kernel: Connection attempt (UNPRIV): \
      IN=ppp0 OUT= MAC= SRC=xx.xxx.xxx.xxx DST=xxx.xxx.xxx.xx LEN=48 \
      TOS=0x00 PREC=0x00 TTL=109 ID=28712 DF PROTO=TCP SPT=4194 DPT=15118 \
      WINDOW=65535 RES=0x00 SYN URGP=0 

which is pretty slick. It explains what it is it saw. "UNPRIV" means the destination port (DPT) is higher than port 1024 (see /etc/services).

Other interesting stuff:

   Apr 18 20:22:42 localhost kernel: Possible DRDOS TCP attempt: \
      IN=ppp0 OUT= MAC= SRC=xxx.xxx.xxx.xxx DST=xxx.xxx.xxx.xx \
      LEN=576 TOS=0x00 PREC=0x00 TTL=63 ID=48819 DF PROTO=TCP SPT=110 \
      DPT=4345 WINDOW=5840 RES=0x00 ACK URGP=0 

That came in on pop3/TCP ("grep 110 /etc/services") looking for some port I don't have (4345).

Among the things you get in AIF, Arno includes "fwfilter", a shell script you can use to massage the logfile entries. I'm already running something else called fwlogwatchviii, but I decided to try this anyway. Arno supplies instructions inside the script explaining how it's to be used. I created a short script in /etc/cron.daily/fwfilter:

   #! /bin/sh
   #  /etc/cron.daily/fwfilter - arno's iptables-firewall activity
   #  monitoring script.

   DAY="$(/bin/date '+%b %e' --date=yesterday)"
   if [ -f "${FWFILTER}" -a -f "${FWLOG}" ]; then
      /bin/grep "${DAY}" ${FWLOG} | ${FWFILTER}

When the system runs its daily cron jobs, this slurps in the firewall log looking for entries from yesterday, pipes them through to Arno's fwfilter script, and the output is mailed to root. The result is quite pretty and informative. Log file entries are colorized by type. The mutt mail program can show ANSI escape sequence color with "set allow_ansi=yes", but you really ought to see the output in (eg.) rxvt or xterm.

Note, depending on your installed version of awk, you may need to fiddle with the fwfilter script prior to using it. See the variable "AWK_BIN". If "which awk" (or better, "dpkg -l | grep awk", or "rpm -qf $(which awk)") mentions "mawk", you'll need to install gawk for fwfilter to work.


I think it was a success. I haven't lost anything, other than a little time. All the things I could do before, I can still do now. I see log records of other, more esoteric events, that I hadn't noticed before. That's the sort of thing I do want to hear about.

I've since had the chance to drop this into the computer I'm rebuilding for a friend as well. It's worked pretty well, hasn't interfered with the two computers' ability to communicate with each other, and was easy to implement. So far, I've nothing but praise for AIF. It's just what I've been hunting for for a long time.

Arno's system is easy to use, can probably scale to fit any situation, and can do it in the hands of a relative newcomer to firewalls. If you can read, you can benefit from Arno's firewall.

If you do try this, and you like it as much as I do, take his advice in the README and donate to his favourite charities.

Rock on, Arno. Bravo. Thank you very much!

 [i] Yes, I do know exactly when it was that I ran across a mention of
    AIF.  I have a short perl script that saves, datestamps, and
    formats as htmls any cuts+pastes that I do during my surfing
    activities ( here).

[ii] My script implementing my old, hand-crafted rules is here.
[iii] The official URL for fauxident is: http://www.alcyone.com/software/fauxident/ Its author is Erik Max Francis http://www.alcyone.com/max/
[iv] MaraDNS is by Sam Trenholme http://www.samiam.org
[v] Arno's iptables-firewall: http://rocky.molphys.leidenuniv.nl/ Freshmeat homepage: http://freshmeat.net/projects/iptables-firewall/?topic_id=151 Arno's mailing list: https://lists.btito.net/mailman/listinfo/firewall
[vi] This happened to me with XCDRoast. All of a sudden, I could no longer do backups. That's why I now make my own archives with afio, use mkisofs to stuff them into an ISO image, followed by cdrecord to burn that to a CD.
[viii] fwlogwatch is from Boris Wesslowski , RUS-CERT http://cert.uni-stuttgart.de
[ix] Debian "stable", at this time (Woody), installs with a 2.2.x kernel. iptables demands 2.4 and up, and the old version of Arno's firewall did too. The latest appears to handle either.
[x] AIF thinks external interface (the Internet) and internal interface (your LAN). AIF only armours the external. It doesn't do much of anything to the internal interface other than making sure anything from it is allowed out. So, testing whether eth0 is up before taking down the firewall is likely the wrong thing to do. I've since disabled that test to verify.
[xi] Of course, everyone knows that "DR-DOS" means "Digital Research Disk Operating System" (a la Gary Kildall), right?


In 1990, after about a decade of processing seismic data (first as a computer operator, then as geophysical technician), I sat down to my first personal computer and started teaching myself how to program in C. After a couple of months at it, I decided it would make a fairly challenging and esthetically pleasing career, so I signed up for a programming course. I graduated with an A average.

Since then, work has taken me from Grande Prairie, Alberta to Khartoum, Sudan. I've worked with SunOS, Solaris, HP-UX, AIX, FreeBSD and OpenBSD. I've worked with many different distributions of Linux (SLS, Slackware, Debian (twice), Redhat, and SuSE). I generally program in Perl. I specialize in generic Unix.

Copyright © 2005, S. Keeling. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

Introduction to Shell Scripting, part 4

By Ben Okopnik

A Blast from the Past! Originally published in Issue 55 of Linux Gazette, May 2000

There are two major products that come out of Berkeley: LSD and UNIX. We
don't believe this to be a coincidence.
 -- Jeremy Anderson

The Deep, Dark Secrets of Bash

Deep within the "bash" man page lurk terrible things, not to be approached by the timid or the inexperienced... Beware, Pilgrim: the last incautious spelunker into these mysterious regions was found, weeks later, muttering some sort of strange incantations that sounded like "nullglob", "dotglob", and MAILPATH='/usr/spool/mail/bfox?"You have mail":~/shell-mail?"$_ has mail!"'

(He was immediately hired by an Unnamed Company in Silicon Valley for an unstated (but huge) salary... but that's beside the point.)

<Shrug> What the heck; I've already gone parasailing and scuba-diving this month (and will shortly be taking off on a 500-mile sail up the Gulf Stream); let's keep living La Vida Loca!

Parameter Expansion

The built-in parsing capabilities of "bash" are rather minimal as compared to, say, Perl or AWK; in my best estimate, they're not intended for serious processing, just "quick and dirty" minor-task handling. Nevertheless, they can be very handy for that purpose.

As an example, let's say that you need to differentiate between lowercase and capitalized filenames in processing a directory - I ended up doing that with my backgrounds for X, since some of them look best tiled, and others stretched to full-screen size (file size wasn't quite a good-enough guide). I "capped" all the names of the full-sized pics, and "decapped" all the tiles. Then, as part of my random background selector, "bkgr", I wrote the following:


# We need _just_ the filename
fn=$(basename $fnm)

# Set the "-max" switch if the test returns 'true'
[ -z "${fn##[A-Z]*}" ] && MAX="-max"

# Run "xv" with|without the "-max" option based on the test result
xv -root -quit $MAX $fnm &


Confusing-looking stuff, isn't it? Well, part of it we already know: the [ -z ... ] is a test for a zero-length string. What about the other part, though?

In order to 'protect' our parameter expansion result from the cold, cruel world (e.g., if you wanted to use the result as part of a filename, you'd need the 'protection' to keep it separate from the trailing characters), we use curly brackets to surround the whole enchilada -

$d is the same as${d} - except that the second variety can be combined with other things without losing its identity, like so:


# "Digitize"
echo ${d}ize

# "Digital"
echo ${d}al

# "Digits"
echo ${d}s

# "Digitalis"
echo ${d}alis

Now that we have it isolated from the world, friendless and all alone... oops, sorry - that's "shell script", not "horror movie script" - I lose track once in a while... Anyway, now that we've separated the variable out via the curly braces, we can apply a few tools incorporated in Bash to perform some basic parsing. Note that I'm going to show the result after each 'echo' statement as if that statement had been executed.

### Let's pick a nice, longish word to play with.

### ${#var} - return length
echo ${#var}

### ${var#word} - cut shortest match from start
echo ${var#*n}

### ${var##word} - cut longest match from start
echo ${var##*n}

### ${var%word} - cut shortest match from end
echo ${var%n*}

### ${var%%word} - cut longest match from end
echo ${var%%n*}

### ${var:offset} - return string starting at 'offset'
echo ${var:7}

### ${var:offset:length} - return 'length' characters starting at 'offset'
echo ${var:1:3}

### ${var/pattern/string} - replace single match
echo ${var/amanuen/paralip}

### ${var//pattern/string} - replace all matches
echo ${var//a/A}

(For the last two operations, if the pattern begins with #, it will match at the beginning of the string; if it begins with %, it will match at the end. If the string is empty, matches will be deleted.)

There's actually a bit more to it - things like variable indirection, and parsing arrays - but, gee, I guess you'll just have to study that man page yourself. <grin> Just consider this as motivational material.

So, now that we've looked at the tools, let's look back at the code -[ -z "${fn##[A-Z]*}" ] Not all that difficult anymore, is it? Or maybe it is; my thought process, in dealing with searches and matches, tends to resemble pretzel-bending. What I did here - and it could be done in a number of other ways, given the above tools - is match for a max-length string (i.e., the entire filename) that begins with an uppercase character. The [ -z ... ] returns 'true' if the resulting string is zero-length (i.e., matches the [A-Z]* pattern), and $MAX is set to "-max".

Note that, since we're matching the entire string, ${fn%%[A-Z]*} would work just as well. If that seems confusing - if all of the above seems confusing - I suggest lots and lots of experimentation to familiarize yourself with it. It's easy: set a parameter value, and experiment, like so -

Odin:~$ experiment=supercallifragilisticexpialadocious
Odin:~$ echo ${experiment%l*}
Odin:~$ echo ${experiment%%l*}
Odin:~$ echo ${experiment#*l}
Odin:~$ echo ${experiment##*l}
...and so on. It's the best way to get a feel for what a certain tool does; pick it up, plug it in, put on your safety glasses and gently squuueeeze the trigger. Observe all safety precautions as random deletion of valuable data may occur. Actual results may vary and will often surprise you.

Parameter State

There are times - say, in testing for a range of error conditions that set different variables - when we need to know whether a specific variable is set (has been assigned a value) or not. True, we could test it for length, as I did above, but the utilities provided by "bash" for the purpose provide convenient shortcuts for such occasions.

### ${parameter:-word} - If parameter is unset, substitute "word"
echo ${joe:-mary}
echo $joe

### ${parameter:?word} - Display "word" or error if parameter is unset
echo ${joe:?"Not set"}
bash: joe: Not set
echo ${joe:?}
bash: joe: parameter null or not set

### ${parameter:=word} - If parameter is unset, set it to "word"
echo ${joe:=mary}
echo $joe

### ${parameter:+word} - "word" is substituted only if parameter is set
echo ${joe:+blahblah}

Array Handling

Another built-in capability of "bash", a basic mechanism for handling arrays, allows us to process data that needs to be indexed, or at least kept in a structure that allows individual addressing of each of its members. Consider the following scenario: if I have a phonebook/address list, and want to send my latest "Sailor's Newsletter" to everyone in the "Friends" category, how do I do it? Furthermore, say that I also want to create a list of names of the people I sent it to, or do some other processing... i.e., it becomes necessary to split it up into fields by length - and arrays become one of the very few viable options.

Let's look at what this might involve. Here's a clip of a notional phonebook to be used for the job:

Name Category Address Email
Jim & Fanny Friends Business 101101 Digital Dr. LA CA fr@gnarly.com
Fred & Wilma Rocks friends 12 Cave St. Granite, CT shale@hill.com
Joe 'Da Fingers' Lucci Business 45 Caliber Av. B-klyn NY tuff@ny.org
Yoda Leahy-Hu Friend 1 Peak Fribourg Switz warble@sing.ch
Cyndi, Wendi, & Myndi Friends 5-X Rated St. Holiday FL 3cuties@fl.net

Whew. This stuff obviously needs to be read in by fields - word counting won't do; neither will a text search. Arrays to the rescue!

# 'nlmail' sends the monthly newsletter to friends listed
# in the phonebook

# "bash" would create the arrays automatically, since we'll
# use the 'name[subscript]' syntax to load the variables -
# but I happen to like explicit declarations.

declare -a name category address email

# A little Deep Hackery to make the 'for' loop read a line at a time
for line in $(cat phonelist)
    # Increment the line counter

    # Load up the 'name' variable
    # etc. for 'category',
    # etc. for 'address',
    # etc. for 'email'...

# Undo the line-parsing magic

At this point, we have the "phonelist" file loaded into the four arrays that we've created, ready for further processing. Each of the fields is easily addressable, thus making the stated problem - that of e-mailing a given file to all my friends - a trivial one (this snippet is a continuation of the previous script):
for y in $(seq $x)
    # We'll look for the word "friend" in the 'category' field,
    # make it "case-blind", and clip any trailing characters.
    if [ -z "${category[$y]##[Ff]riend*}" ]
        mutt -a Newsletter.pdf -s 'S/V Ulysses News, 6/2000' ${email[$y]}
        echo "Mail sent to ${name[$y]}" >> sent_list.txt
That should do it, as well as pasting the recipients' names into a file called "sent_list.txt" - a nice double-check feature that lets me see if I missed anyone.

The array processing capabilities of "bash" extend a bit beyond this simple example. Suffice it to say that for simple cases of this sort, with files under, say, a couple of hundred kB, "bash" arrays are the way to go.

Note that the above script can be easily generalized - as an example, you could add the ability to specify different phone-lists, criteria, or actions, right from the command line. Once the data is broken up into an easily-addressable format, the possibilities are endless...

Wrapping It Up

Bash, besides being very capable in its role as a command-line interpreter/shell, boasts a large number of rather sophisticated tools available to anyone who needs to create custom programs. In my opinion, shell scripting suits its niche - that of a simple yet powerful programming language - perfectly, fitting between command-line utility usage and full-blown (C, Tcl/Tk, Perl, Python) programming, and should be part of every *nix user's arsenal. Linux encourages the "do it yourself" attitude among its users by giving them access to powerful tools and the means to automate their usage - something that I consider a tighter integration (and that much higher a "usability quotient") between the underlying power of the OS and the user environment.

Until next month -
Happy Linuxing!

Quote Of The Month:
"...Yet terrible as UNIX addiction is, there are worse fates. If UNIX is
the heroin of operating systems, then VMS is barbiturate addiction, the Mac
is MDMA, and MS-DOS is sniffing glue. (Windows is filling your sinuses with
lucite and letting it set.)

You owe the Oracle a twelve-step program."
 -- the Usenet Oracle


The "man" pages for 'bash', 'builtins', 'sed', 'mutt'

"Introduction to Shell Scripting - The Basics" by Ben Okopnik, LG #53 "Introduction to Shell Scripting" by Ben Okopnik, LG #54 "Introduction to Shell Scripting" by Ben Okopnik, LG #55 "Introduction to Shell Scripting" by Ben Okopnik, LG #56

picture Ben is the Editor-in-Chief for Linux Gazette and a member of The Answer Gang.

Ben was born in Moscow, Russia in 1962. He became interested in electricity at the tender age of six, promptly demonstrated it by sticking a fork into a socket and starting a fire, and has been falling down technological mineshafts ever since. He has been working with computers since the Elder Days, when they had to be built by soldering parts onto printed circuit boards and programs had to fit into 4k of memory. He would gladly pay good money to any psychologist who can cure him of the recurrent nightmares.

His subsequent experiences include creating software in nearly a dozen languages, network and database maintenance during the approach of a hurricane, and writing articles for publications ranging from sailing magazines to technological journals. After a seven-year Atlantic/Caribbean cruise under sail and passages up and down the East coast of the US, he is currently anchored in St. Augustine, Florida. He works as a technical instructor for Sun Microsystems and a private Open Source consultant/Web developer. His current set of hobbies includes flying, yoga, martial arts, motorcycles, writing, and Roman history; his Palm Pilot is crammed full of alarms, many of which contain exclamation points.

He has been working with Linux since 1997, and credits it with his complete loss of interest in waging nuclear warfare on parts of the Pacific Northwest.

Copyright © 2005, Ben Okopnik. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

Measure CPU Fan Speed with an RTAI/LXRT Stroboscope!

By Pramode C.E.

The BIOS on most modern motherboards will show you the rotational speed of the CPU fan. Let's design a simple ‘optical’ experiment for verifying whether the value reported is indeed correct. For maximum voodoo effect, the experiment should be conducted at night with the only source of light being an LED blinking under the control of some dark and devious RTAI/LXRT code!

What is the big idea?

Think of a fan with four blades rotating at a speed of 3600 rotations per minute (RPM.) Let's mark a part of one of the blades with a distinct color, say white. Once the fan starts rotating, it will be difficult for us to differentiate this blade as everything becomes blurred. Each full rotation takes 1/60th of a second. What if we look at the blades for a very brief period only once in that 1/60th of a second? It will appear as if the white mark is motionless because each time we will see it at almost the same position each revolution. A light source (like an LED) can be made to generate short bursts of light at fairly high frequencies; an object illuminated by such a source will be seen only during those periodic short bursts of light. When the frequency of the bursts match the frequency of rotation of the object being observed, the object appears to be stationary. This is the principle of the stroboscope.

The hardware

There is almost always a cooling fan attached to the CPU in a desktop computer; your only job is to ‘mark’ one of the blades. I did this by pasting a small piece of white paper onto one blade and it seems to work well.

[Editor's note - leaving a high-speed fan unbalanced will shorten the life of its bearings. I strongly advise you to remove the marker from the fan when you are done with the experiment! — dsrich]

The light source is a ‘high-brightness’ LED controlled by the parallel port. It's better that the LED is driven by a transistor other than the parallel port output to increase the amount of current through the LED and therefore its brightness. If you are new to PC interfacing, please proceed with caution, preferably after reading the material presented on web sites such as this one.

The software

Generating small (and precise) time delays under a general purpose operating system like Linux is not an easy job. In one of my earlier articles, I had explained the use of an exciting technology called ‘Real Time Application Interface’ (RTAI) which converts GNU/Linux into a hard real-time OS capable of doing complex timing-sensitive control operations. The trouble with the ‘classical’ RTAI approach was that timing sensitive code had to be written as modules and loaded into the kernel. The more ‘modern’ way of doing things is to use a component of RTAI called LXRT and code real-time tasks as ordinary POSIX threads.

Setting up RTAI/LXRT

RTAI/LXRT is composed of the following:

Here are the steps required to get RTAI/LXRT running on your machine:

  1. Download a a ‘plain’ Linux kernel (from kernel.org) - I used linux-2.6.10.
  2. Download rtai-3.2-test3 from here
  3. When you untar the RTAI source package, you will see several patches under the base/arch/i386/patches directory. Apply the proper patch to your newly downloaded Linux kernel (hal-linux-2.6.10-i386-r9.patch, in case you are using Linux 2.6.10). The patch should apply cleanly.
  4. Configure and compile the kernel. When you configure the kernel, make sure that module versioning support is disabled. The experimental ‘pass parameters in registers’ feature should also be disabled. You can verify both by reading your .config file (after you have done make menuconfig) which should contain the lines:
    # CONFIG_REGPARM is not set
    # CONFIG_MODVERSIONS is not set
  5. Configure and compile rtai-3.2-test3. The configuration and compilation process is fairly simple - you just have to do:
    make menuconfig; make; make install
    When you run make menuconfig, you will be asked to provide the full path to the RTAI-patched Linux kernel source tree. During make install, a few directories will be created under /usr/realtime/ - the header files, libraries and kernel modules required to do RTAI/LXRT development will be found under these directories.
  6. Reboot with the newly compiled kernel. Go to the /usr/realtime/modules directory and try to insert the modules rtai_hal.ko and rtai_lxrt.ko:
    cd /usr/realtime/modules/
    insmod ./rtai_hal.ko
    insmod ./rtai_lxrt.ko
    The modules should load cleanly. Check the output of the program dmesg if there is any problem.

Blinking LEDs

The traditional ‘hello world’ program of the hardware enthusiast is a simple LED blinker. Let's code it as an RTAI/LXRT application:

[Download Listing 1]

/* led1.c, an LXRT LED blinker */

#include <pthread.h>
#include <rtai_lxrt.h>
#include <sys/mman.h>
#include <sys/io.h>

// delay in nanoseconds
#define TICKS 500000000 

        RT_TASK *task;
        int priority = 0, i;
        int stack_size = 4096;
        int msg_size = 0; // use default

        // get enough privilege to 
        // access the I/O ports.


        task = rt_task_init(nam2num("main"), 
                        priority, stack_size, msg_size);
        if(task == 0) exit(1);

        for(i = 0; i < 10; i++) {
                outb(0xff, 0x378);
                outb(0x0, 0x378);
        // back to non-rt land!
        return 0;

The rt_task_init function registers our program with the RTAI/LXRT kernel subsystem - the return value is a pointer. Each task should have a name associated with it - we have assigned the name main to our task.

Timing delays in RTAI/LXRT are generated by using the hardware timer chip found on the motherboard. The timer is capable of generating interrupts and can run in two modes - periodic and one shot. When running in periodic mode, it keeps generating interrupts at a fixed rate (say once every 1 millisecond). If all the time delays required in your code can be expressed as multiples of a ‘base’ period (say 1 millisecond), you can think of using the periodic mode. In the ‘one-shot’ mode, the timer will generate just one interrupt at a time after a certain delay. This mode provides more flexibility as the timer can be reprogrammed repeatedly to fire at differing intervals.

The mlockall function is called to disable paging of program memory - paging might result in the program being transferred to persistent swap store medium thereby affecting its real-time behavior.

The rt_make_hard_real_time function transfers control of our application to the RTAI/LXRT realtime scheduler. The effect of this transition is that our code will never get disturbed by normal Linux processes or activities going within the Linux kernel. Note that once the program becomes ‘hard real time’, it should never invoke Linux system calls - doing so will result in the task becoming non real-time.

The nano2count function converts time specified in nanoseconds to some ‘internal count units’ specific to RTAI/LXRT. We have defined TICKS to be 500 milliseconds; so


will suspend our task for half a second. If we run the code with an LED connected to an output pin (pins 2 to 9) of the parallel port, we will see it blinking on and off ten times.

The program should be compiled like this:

cc led1.c -I/usr/realtime/include -L/usr/realtime/lib -lpthread -llxrt

Before running the code, make sure to edit your /etc/ld.so.conf file so that it contains the path to the directory containing the LXRT library (say /usr/realtime/lib); the ldconfig command should also be executed.

The LXRT stroboscope!

I enter the CMOS setup program of my Athlon64 machine and note that my CPU fan is running at a speed of about 3900 rotations per minute, or 65 rotations per second. The time for one rotation is therefore (1000000000/65) nanoseconds, or 15384615 nanoseconds. The idea is to blink an LED connected to pin number 2 of the parallel port at this frequency.

LED on + off time = 15384615 nanoseconds
LED on time = 15384615/12 nanoseconds

Out of the total period of 15384615 nanoseconds, the LED would stay on for (1/12)th of the period and would be off for the remaining time. The fan (and the white paper stuck onto one of the blades) will be visible only during this brief period; by the time the LED is on again, the blade with the paper will have returned back to its original position. Thus, we should be able to see an almost static image of the paper pasted onto the blade.

In practice, I had to continuously tweak the period to get an almost static image. This might be because the fan speed is not remaining constant and keeps on changing slightly.

The code I had written is shown below:

[Download Listing 2]

#include <pthread.h>
#include <rtai_lxrt.h>
#include <sys/mman.h>
#include <sys/io.h>

#define ONE_ROTATION   15384615

volatile int stop = 0;
unsigned int on_time, off_time; 

void* led_task(void *arg)
        RT_TASK *task;
        int priority = 0, i;
        int stack_size = 4096;
        int msg_size = 0;

        printf("LED started ...\n");

        task = rt_task_init(nam2num("led"), 
                        priority, stack_size, msg_size);
        if(task == 0) exit(1);

        while(!stop) {
                outb(0xff, 0x378);
                outb(0x0, 0x378);
        return 0;

        RT_TASK *task;
        int priority = 10, i;
        int stack_size = 4096;
        int msg_size = 0;
        pthread_t t1;

        on_time = ONE_ROTATION/12;
        off_time = ONE_ROTATION - on_time;

        task = rt_task_init(nam2num("main"), 
                        priority, stack_size, msg_size);
        if(task == 0) exit(1);
        pthread_create(&t1, 0, led_task, 0);
        stop = 1;
        pthread_join(t1, 0);
        return 0;

Note that the main thread is spawning a new thread which calls rt_make_hard_real_time - this is the real-time LED flasher. The main thread remains non real-time because it has to interact with Linux (the getchar function will call the read system call). This is typical of LXRT code; one or more non real-time threads will do all the Linux-interaction part and the hard real-time threads will do some hardware manipulations without invoking any system calls.


As I mentioned in the beginning, do this experiment at night, preferably near around midnight! Whether you see the stroboscopic effect or not, your friends (or family members) who might catch you in action staring at the open motherboard of the machine in pitch darkness with an LED light in your hands are sure to get a very favorable impression about your general geek qualities!

If you have trouble compiling your RTAI patched Linux kernel, you might try using my .config file (Linux 2.6.10, patched with the corresponding RTAI patch.) You should modify the CPU type to match your computer before compilation.

You can learn more about RTAI/LXRT by reading the paper Porting your C++ GNU/Linux application to RTAI/LXRT. The LXRT-Informed FAQ is good reading too. The wikipedia entry on strobe light links the phenomenon to ‘aliasing’ which is common in digital signal processing.

As we are not dealing with sub-millisecond time periods here, the usual Linux delay mechanisms (based on the system clock running at 1KHz on the 2.6 kernel) should be sufficient for this experiment. Doing so is left as an exercise to the reader!

[BIO] I am an instructor working for IC Software in Kerala, India. I would have loved becoming an organic chemist, but I do the second best thing possible, which is play with Linux and teach programming!

Copyright © 2005, Pramode C.E.. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005


By Shane Collinge

These images are scaled down to minimize horizontal scrolling. To see a panel in all its clarity, click on it.

[cartoon] [cartoon] [cartoon]

All HelpDex cartoons are at Shane's web site, www.shanecollinge.com.

[BIO] Part computer programmer, part cartoonist, part Mars Bar. At night, he runs around in a pair of colorful tights fighting criminals. During the day... well, he just runs around. He eats when he's hungry and sleeps when he's sleepy.

Copyright © 2005, Shane Collinge. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005


By Jon "Sir Flakey" Harsem

These images are scaled down to minimize horizontal scrolling. To see a panel in all its clarity, click on it.

[cartoon] [cartoon] [cartoon]

All Qubism cartoons are here at the CORE web site.

[BIO] Jon is the creator of the Qubism cartoon strip and current Editor-in-Chief of the CORE News Site. Somewhere along the early stages of his life he picked up a pencil and started drawing on the wallpaper. Now his cartoons appear 5 days a week on-line, go figure. He confesses to owning a Mac but swears it is for "personal use".

Copyright © 2005, Jon "Sir Flakey" Harsem. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

PmWiki - Wiki the Painless Way

By Raj Shekhar

1.1 Why use a Wiki

In the Issue 108 of Linux Gazette, Neil Youngman had written an article introducing JSPWiki. In this article I would like to introduce you to an obscure but feature packed Wiki, called PmWiki. Before diving into the intricacies of Wiki, I will speak a bit about where I have used wikis and found them quite good.

Wikis make it easy to create interlinked HTML pages. You can concentrate on the task of writing actual content without worrying about HTML tags or how to create the navigation links. The easy markup of the wikis make it very easy to copy text from emails, chat logs and random web pages and add it in your wiki. The wikiengine takes care of transforming the simple text to clean HTML. (a side note: you should give credit to the original author when you are copying text from some email/web page.)

Most wikiengines come with a search engine too, which makes the task to searching through the Wiki pages easier.

Available anywhere to-do list
I have tried a lot of things to keep a list of things I have promised to do. At one time I experimented with keeping it on a wiki page (on the Internet) and then kept adding/subtracting to it. (these days I use the excellent emacs-planner ). Since it is quite easy to edit the wiki page, it was quite effective.

Keeping meeting notes
In a brainstorming session, it is important to keep notes. However, if the company is small, the onus is upon the participants to keep track of the discussions that happened. In my previous company (which was quite small), each one of us kept our own notes in a simple text file. After the session was over, we used to create one wiki page and put our individual notes on that page.

1.2 Why Pmwiki

The main features that I was looking for in the Wiki were Looking through the list of available Wiki options, I found PmWiki. It met all the requirements that I was looking for. I also saw that this project definitely believed in eating its own dogfood. The documentation as well as the bug list has been maintained using PmWiki itself. I also had a look at the wiki's markup syntax, which seemed to be simple enough.

1.3 Installation

The Installation page has instructions on how you can get the latest version of PmWiki. The page also has installation instructions. As it is common with most of the PHP applications, the process is quite simple. You have to copy the files to a directory in your webserver's document root (for example in /var/www/html/mywiki) and then change the directory permissions to 2777. Use a browser to open the pmwiki.php page in this directory i.e. `www.yourdomain.com/mywiki/pmwiki.php'. If the permissions have been set properly, you should see the default PmWiki home page.

1.4 Customization

Once the wiki has been installed, the next step is to customize it. The customizations are usually controlled by the config.php file found in the local directory. When you first install PmWiki, the config.php file will not exist and so you'll have to create one if you want to perform any local customizations. You can copy the sample-config.php file (in the same directory as pmwiki.php) to local directory and use it as a starting point.

The first thing that you may want to customize is the $WikiTitle. The $WikiTitle variable gives the name of your site as it will appear in a user's browser title bar.

The next thing you may want to customize is the skin. If you lack design skills, like me, there is still hope for you. The PmWiki has good skins available at the site itself. Once you have downloaded the skin, unzip and put it in the folder pub/skins/). Then you set the $Skin variable to the name of the directory that contains your skin files. For example, if you like JHSkin, download the zip file and unzip its content. You will have a folder called jhskin. Upload this folder to the pub/skins folder in your wiki's directory. Then change the Skin variable to $Skin = 'jhskin'. When you reload the wiki in your browser, you will see that it sports a new look now.

The PmWiki cookbook has tips for more customizations.

1.5 Antispam Measures

Since wiki pages are editable by everyone, spammers use them to serve as their link farms (For example, this google search will help you to locate some abused wikis). Luckily PmWiki provides a few measures to fight against this menace. You can use the Movable Type black list to help control wiki spam. If the number of users who need to use the wiki is small, you can even give them indivual passwords. There is a UserAuth script available too, which allows you to manage a larger number of users. This script also allows you to register users before they can edit pages.

If you are making a wiki which will reside on your LAN, you can leave out these antispam measures. If you are making the wiki for your own use, I strongly suggest you have an admin password and allow only the admin to edit the pages. If you are making a world editable Wiki, be sure to review the PmWiki Security page.

1.6 Conclusion

Now that you are ready to run your own wiki, here are some helpful links -


I work for Yahoo! Bangalore (and I think it is the best place to work :-) ) as an Operations Engineer.

I am a staunch supporter of Free Software and the No Software Patents campaign. In my free time, I try to keep a semi-regularly updated blog.

Copyright © 2005, Raj Shekhar. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

The Foolish Things We Do With Our Computers

By Ben Okopnik

"Foolish Things" is a now-and-again compilation that we run based on our readers' input; once we have several of these stories assembled in one place, we get to share them with all of you. If you enjoy reading these cautionary tales of woe, proud stories of triumph, and just plain weird and fun things that happen between humans and silicon, that's great; if you have some to share so that others may enjoy them, even better. Please send them to .

[ You can even tell us that it happened to A Friend of Yours, and we'll believe you. ]

-- Ben

Teacher, Teach Thyself

from The Perl-Trainers list

Ben Okopnik wrote:
> On Wed, Dec 22, 2004 at 05:58:00PM -0800, Randal L. Schwartz wrote:
> > >>>>> "Tim" == Tim Maher writes:
> >
> > >> On Wed, Dec 22, 2004 at 08:33:25PM -0000, Peter Scott wrote:
> > >> Some advice for Perl trainers doing hands-on demonstrations:
> > >> Don't type "my $self = shift" (or shift anything else) without
> > >> practice.  It is too easy to typo and leave the 'f' out of
> > >> 'shift'.  For some reason I am more prone to making this
> > >> mistake in front of an audience than when coding alone.
> > >> Peter Scott
> >
> > Tim> Been there; done that!  I also recommend avoiding attempts to
> > Tim> type the potentially much more embarrassing "$count" for the same
> > Tim> general reason (did it once; won't ever take the chance again!).
> >
> > And don't do live demos on your laptop where you might have personal
> > data unless you've already rehearsed every step of your demo in the
> > privacy of your own cubicle, and verified that nothing of questionable
> > interest might be exposed.
> >
> > And on a completely unrelated topic, keep in mind that the resource
> > fork of a movie you downloaded to a UFS partition on MacOSX will be
> > stored in a file named "._moviename.mpg", which doesn't get removed
> > when you say "rm *".
> >
> > :-)
> Not that I've ever suffered from any of the above problems, oh no...
> (Displaying your /etc/bashrc, when it happens to contain aliases
> designed to discourage those who type curse words as commands - during a
> demo at a bank headquarters, no less - cannot *possibly* have happened
> to me. Never.)

Oh, Fuss and Botheration. I think I've Just Nuked the World...

from Richard Neill

When I was learning Linux, I decided to install the recent version of Mozilla (from mozilla.org), since the version that came with Mandrake 8.0 was a few months old, and Moz was really making leaps every month (around version 0.93, I recall).

Anyway, the installer asked me where I wanted to put it. I looked where the distro put it, (/usr/bin/mozilla), and told the installer to put the new copy in /usr/bin. Of course the installer wanted to put a whole bunch of files there, not just the mozilla executable, and it complained that the directory wasn't empty. Might it delete the other files in the directory, it asked? Yes, I said.

Bother. It's surprising how useless a Linux distro gets when you effectively do rm -rf /usr/bin. It's also odd that nothing immediately breaks, e.g. KDE continued to work fine.

What I should have done was rpm -Va and then reinstalled the packages. But in the end, I did a full reinstall.

If It Walks Like a DIG, and quacks like a DOC...

from Robin Twombly

I run several version of Linux and the worst problem I had was when I miss typed a directory name when deleting. I wanted to delete the folder var/www/ htdig and I accidentally typed var/www/htdocs thus deleting all my internet files for my apache web server.

At the time I was not clear on how to undelete a file on Linux.

Thank you for many hours of fun reading.
Happy Linux Gazette Reader

[ A pleasure, Robin. Thanks for contributing to Making Linux Just a Little More Fun! -- ben ]

picture Ben is the Editor-in-Chief for Linux Gazette and a member of The Answer Gang.

Ben was born in Moscow, Russia in 1962. He became interested in electricity at the tender age of six, promptly demonstrated it by sticking a fork into a socket and starting a fire, and has been falling down technological mineshafts ever since. He has been working with computers since the Elder Days, when they had to be built by soldering parts onto printed circuit boards and programs had to fit into 4k of memory. He would gladly pay good money to any psychologist who can cure him of the recurrent nightmares.

His subsequent experiences include creating software in nearly a dozen languages, network and database maintenance during the approach of a hurricane, and writing articles for publications ranging from sailing magazines to technological journals. After a seven-year Atlantic/Caribbean cruise under sail and passages up and down the East coast of the US, he is currently anchored in St. Augustine, Florida. He works as a technical instructor for Sun Microsystems and a private Open Source consultant/Web developer. His current set of hobbies includes flying, yoga, martial arts, motorcycles, writing, and Roman history; his Palm Pilot is crammed full of alarms, many of which contain exclamation points.

He has been working with Linux since 1997, and credits it with his complete loss of interest in waging nuclear warfare on parts of the Pacific Northwest.

Copyright © 2005, Ben Okopnik. Released under the Open Publication license unless otherwise noted in the body of the article. Linux Gazette is not produced, sponsored, or endorsed by its prior host, SSC, Inc.

Published in Issue 114 of Linux Gazette, May 2005

The Linux Launderette


(?)Microsoft buzz of the day
(?)For the Back Page
(?)Spam, spam, spam, spam...
(?)Hacker high school
(?)startx problem ?
(?)For Ben, courtesy of the Spammers
(?)[Sussex] A little bash help...
(?)Industry News
(?)Amusing note on security...
(?)Strange 419 spam
(?)Hitchhiker's guide trailers
(?)419 spam
(?)Re: Pouding
(?)Investment Relationship /waiting to hear from you.
(?)Software patents
(?)Register news
(?)Dave Allen (was Re: [TAG] UPS Problems..)
(?)Cron Jobs not being run
(?)Question about file mtime on linux
(?)PAM auth failure - can't login in.
(?)My PC is slower than yours...
(?)need information about sylpheed-claws
(?)Spam jokes
(?)Re: CD/RW's on Fedora Core 3
(?)India currency question (/Flash sucks)
(?)Hydrogen power
Greetings, greetings, greetings. After two months with sporadic internet access, I managed to grab the gems of wit and general non-linux-related chat that make up this section of LG. Enjoy.
Just before I was disconnected I had been having a conversation with a reader about music related things, and was unable to reply. I tried to sneak it through to last month's issue, but Ben rejected it due to the lack of Linux related content. I would have liked to have attached it to one of my "Songs in the Key of Tux" articles, but since I think the reply is long overdue, I've put it here.
As I've mentioned, I'm not currently connected to the 'net, so some of this month's launderette may have appeared in the last, or in TAG. If so, er, sorry.
Also, after several months of drinking way too often and way too much I managed to start writing some new articles that I haven't been able to complete by deadline time, but have been able to provide hints for. Next month!

-- Jimmy O'Regan

(?) Microsoft buzz of the day

From Sluggo

http://www.theregister.co.uk/2005/02/04/reboot_tablet_xp Microsoft urges Tablet users to reboot daily due to a memory-leaking program.

(?) For the Back Page

From Sluggo

% lost svn.wk4py.org
zsh: command not found: lost

(?) Spam, spam, spam, spam...

From Ben

I've just fished this one out of my spambox. The thing that's got me tremendously amused is that they tell you it's a load of baloney... just look at the sender's name!
----- Forwarded message from Tunde Salami <tnd_salami@eresmas.com> -----

From: Tunde Salami <tnd_salami@eresmas.com>
To: tnd_salami@eresmas.com
Reply-To: tnd_salami@box.az
Date: Tue, 12 Apr 2005 19:09:20 GMT
Subject: *****SPAM***** ATTENTION



[ snip ]

----- End forwarded message -----

(?) Hacker high school

From Sluggo

Hacker high school

Now we have a place to send all those losers who write in wanting to know how to hack, especially those who say, "Please tell me how to hack my school's computer."

(Trying not to say, "for those about to hack... we salute you." Argh, Jimmy, how do I get heavy metal out of my brain?

(!) [Jimmy] Heh. I get to the thread too late...
I've been trying to get more metal into my head, so I'm the wrong person to ask. I had a two day hangover after Easter Sunday (woke up in the home of my Polish friends, joined them for a traditional Polish Easter breakfast (which happened to include a bottle of vodka and a bottle of tequila), slept through the rest of the day, woke up thinking it was still Saturday. Y'know, the usual) where all I could do was play guitar. Out of the 16 or 17 more or less completed songs I came up with, only 4 or 5 passed the quality control process before being presented to my band (my next "Songs..." article will have some of the details), but the thing that's really irking me is that every time I try to finish off one of these metal songs, I end up getting delicate acoustic music that'll never be played.
(!) [Ben] Extracting those iron spikes from your head would be a good start...

(?) I gave it a heavy dose of Jean-Michel Jarre over the weekend but it didn't help.)

(!) [Ben] Jean-Michel Jarre??? Oh, well. There was some small chance of recovery up until you did that, but now... it's not even worth attempting. Heavy metal locked in by J-MJ is a self-replicating meme that results in an instantly-fatal case of acne for its host if anyone tries to remove it; you're permanently stuck. Don't worry, though; we'll keep AC/DC, some '70s hair bands, etc., piped into your room at the asylum, and you'll be able to lead some approximation of a normal life.

(?) Ben, please, not the hair metal!!! Anything but that!!! I'm sorry I called you a dirty communist spy. You're the kindest secret agent I've ever met, and those dark sunglasses look really good on you. Your capitalist credentials are impeccable, and your houseboat is the spiffiest in the harbor.

(!) [Ben] Heh, heh, heh. It's just a matter of knowing which threat to use.
Now we know your weak spot, comrade...
(!) [Jimmy] Wow Ben. I'd have never thought of using 70s hair metal. I'd have gone for the much more cruel option of 80s hair metal. Y'know, the hair spray, the wind machines...
Am I Evil? Yes I am.
(But Diamond Head are NWOBHM :)

(?) startx problem ?

From Sluggo

[Not cc'ing querent.]

Thomas Adam wrote:
> Oh, and *please* *fix* your damn .signature file -- be it an
> automatically-generated one, or just plain idiocy on your part -- I'm
> still getting Tux *twice* and you still *haven't* reduced your
> .signature file to <=5 lines.  If you think I'm being harsh -- wait
> until you post elsewhere on some other list;  they'll lynch you more so
> than I.

Watch out. Thomas has been training under Ben Okopnik. You really don't want to see his harsh side.

(!) [Breen] Ah, but has he earned his sunglasses yet?
(!) [Thomas] Ha! You should see the sunglasses I wear... :P
(!) [Jason] Oakley makes some sunglasses with an MP3 player built-in. 128 megs internal flash memory. And it's expandable.
So I'm thinking it's only a matter of time before Moore's Law makes it possible to run Linux on sunglasses, maybe video out via a HUD. How cool would that be?
Officer:	You ran a red light.
You:		Sorry, I was recompiling my kernel.

(?) As cool as my Java doorknob.

(!) [Ben] [ blink ] Say what? Is all that Python coding making you hallucinate *again*?

(?) Back when Sun first publicized Java and I was learning it, there was a common belief that Java would become ubiquidous in embedded systems. That led to the joke about the Java doorknob.

(!) [Ben] But the proto-ancestor of that was...


I went to my first computer conference at the New York Hilton about 20 years ago. When somebody there predicted the market for microprocessors would eventually be in the millions, someone else said, "Where are they all going to go? It's not like you need a computer in every doorknob!" Years later, I went back to the same hotel. I noticed the room keys had been replaced by electronic cards you slide into slots in the doors. There was a computer in every doorknob. -- Danny Hillis


(!) [Ben] Sure, sure, "Java doorknob", "millenium hand and shrimp", the Cabal (There Is No Cabal)... next thing you know, you'll be talking about "we're bringing democracy to Iraq".
Nah, maybe not. Nobody could be crazy enough to believe that.
(!) [John] Gosh, if only that were true ....
(!) [John (again)] I should have included this link!
(!) [Ben] Wow, 20MB worth of video. I tried to download it while I was teaching class, but the bandwidth in that part of the world is better described as "bandnarrowth"...
I'll give it another shot, now that I'm back at my hotel.
(!) [Ben] [sigh] Yeah.
I often wonder if even its proponents believe it. They use it, yeah, but believe it? Brain damage isn't that common.
Not that the first part of that makes any sense to me either, but then I can't fit the words "morality" and "murder for profit" into the same head space...
(!) [Jay] Yep.
Java Doorknob:
http://www.maxim-ic.com/products/ibutton/ibuttons http://www.maxim-ic.com/products/ibutton/ibuttons/java.cfm
Ok. Admittedly, I had to link two web pages to pull it off, but...
(!) [Ben] [laugh] There's always a smart guy in a Linux crowd. I just knew that somebody here would do a "yeah, but if we epoxied a rabid weasel at the tip of each helicopter blade, and applied 14,000.06v to three slices of French toast..."
(!) [Jay] Naw; I think 14KV would blow those things up...
(!) [Ben] (Nice to see you still here, Jay. It's been a month for Voices From The Past.)
(!) [Jay] Good to be back. I think I was actually in one of your necks of the woods last week, actually; I had to make a run to Ormond Beach, over north of Daytona.
(!) [Ben] Damn. You were within 40 minutes or so of me - and I was actually there for once (I've had a metric assload of travel lately, more travel than during the entire last year. QED: I'm writing this from a hotel room in New Mexico.)
(!) [Jay] Missed all the bikini's though.
(!) [Ben] Yes, well... those are maybe a week or two away. There's a bar/marina/restaurant in Salt Run (a part of St. Augustine) that is unbelievably packed with babes in the summer; be sure your heart is in good working order before checking it out.
(!) [Thomas] I have often quipped about this to my lecturer, who is very pro Java. Suffice it to say, he's not keen. :) What really annoys him is when I (seemingly) submit the Ruby equivalent of an answer into my work... :)
(!) [Ben] Well, if I'd known that you wanted to annoy him, all you had to do was tell me...


Learning Java has been a slow and tortuous process for me. Every few minutes, I start screaming 'No, you fools!' and have to go read something from Structure and Interpretation of Computer Programs to de-stress. -- The Cube, www.forum3000.org


(!) [Thomas] Hmm, in other news, it's my birthday. :/
(!) [Thomas] Ah, yes - old age creeping up on you. They say that the first thing to go is memory, and the second is... uh, wait... give me a minute, willya?... I'll get back to you on that one.
(!) [Thomas] I feel old (not as old as Ben -- I don't think that's possible, is it? :P).
(!) [Breen] Oh, yes. It's possible. It's very possible. . .
(!) [Ben] It is, as George Burns said, better than the alternative. :p
(!) [Thomas] But I have finally got a copy of Vol 1 of "The Nuggets" collection. Four CDs of timeless and classic 1960s American psychedelia music. Yay. :) I can finally update my TAG/author bio to reflect this.
(!) [Ben] Hey, cool! That proves you've joined the ranks of the old farts; you even listen to the right kind of music!
(A very happy birthday to you, my friend. May your wishes come true in the way you'd best like them to, and may your life be long and full of love and friendship.)
(!) [Sluggo] Congratulations. How many whacks on the behind do you get?
(!) [Ben] Interesting birthday rituals you have, Mike. Most people just light some candles and have a bit of cake, but to each his own, I guess. BTW, when is your birthday? I've got a pair of perfectly good wooden oars, about 4 feet long and 10 lbs. apiece, that are just going to waste for lack of use... :)
(!) [Sluggo] For your birthday we have... a new version of FVWM! Just kidding, sorry. But check out the spy games Ben installed on the computer in the TAG lounge. Very realistic. There's even one set in Ireland where he's running away from a crime scene near Jimmy's house, and Jimmy puts his hand through the door....
(!) [Ben] SPOILER: Mike did it. In the pantry, at 7:30pm, with a wooden paddle.
(!) [Jimmy] Just for the record, I never put my hand through that door, I only cracked the glass (lucky for me it was reinforced glass, so I still have my hand).
Note, however, that I said that door...

(?) For Ben, courtesy of the Spammers

From DesperateHusbands

Marriage is not a word. It is a sentence - a life sentence.
Marriage is very much like a violin; after the sweet music is over, the strings are attached.
Marriage is love. Love is blind. Therefore, marriage is an institution for the blind.
Marriage is an institution in which a man loses his Bachelor's Degree and the woman gets her Masters.
Marriage is a thing which puts a ring on a woman's finger and two under the man's eyes.
Marriage certificate is just another word for a work permit.
Marriage is not just a having a wife, but also worries inherited forever.
Marriage requires a man to prepare 4 types of "rings": The Engagement Ring, The Wedding Ring, The Suffe-Ring, The Endu-Ring.
Married life is full of excitement and frustration: In the first year of marriage, the man speaks and the woman listens; In the second year, the woman speaks and the man listens; In the third year, they both speak and the neighbors listen.
It is true that love is blind but marriage is definitely an eye-opener.
Getting married is very much like going to the restaurant with friends. You order what you want, and when you see what the other fellow has, you wish you had ordered that.

(?) [Sussex] A little bash help...

From Geoffrey Teale

See attached GNUecho.1.txt

(?) Industry News

From Jimmy O'Regan

>From MillardbrachydomalL7gGv at mailshack.com  Sat Feb  5 16:39:57 2005
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
From: MillardbrachydomalL7gGv at mailshack.com (Vincent M. Bartlett)
Date: Sat Feb  5 16:45:04 2005
Subject: [TAG] Industry News
Message-ID: <RMNN$rw7O6.nXRubWG$tIJ7@applausive3.gawab.com>


13. Ninety six bottles of beer, three a's, three b's, one c, two d's, thirty two e's,
six f's, two g's, six h's, twelve i's, one j, one k, five l's, one m, eighteen n's,
fourteen o's, one p, six r's, twenty seven s's, twenty t's, two u's, seven v's, nine
w's, five x's, and five y's on the wall.
What! No z? No q?

(?) Amusing note on security...

From Ben Okopnik

>From http://securitygeeks.shmoo.com/ :

No vulnerabilities in ...

6 days 18 hours

20 days 59 minutes

170 days 1 hour

292 days 23 hours

(?) Strange 419 spam

From Mike Orr

Comments below.
-------- Original Message --------
Date: 	Thu, 10 Mar 2005 08:45:52 -0800
From: 	presidency1@zipmail.com.br

Committee on Foreign Contract Payment (RESOLUTION

                                            Date: 10h MARCH 2005

ATTN: Sir/Madam

We, the entire members of the Federal House of Senate, on behalf of the
Federal Republic of Nigerian Government under the auspices of the civilian
Head of State, President Olusegun Obasanjo have unanimously decided and
plead for your forgiveness and for you to revert your distrust upon the
entire arms of the government and Nigeria as a body. All efforts are being
put into practice to chastise all our corrupt officials, presenting the
country as scam Nation before the world.

This is a bid to uphold and maintain our international reputation and premise
to the United Nation, OPEC, ECOWAS and the rest of the International Organization
and the World as a whole. There was an inflow in our Nigerian Oil Well Reserve
Trust Account, as a result of the involvement of International Financial
institutions such as UN,IMF,WORLD BANK and PARIS CLUB in the development
of regional economy,sufficient funds have been approved for the settlement
of every foreign firm who executed contract in Nigeria and who are still
yet unpaid. On the receipt of your full outstanding foreign contract amount,we
request that you contact the office of the IMF/World Bank informing them
that you have been paid,for this will help to change our image before the
world/international community.

We strictly instruct and advise that you contact only The Director,FIDELITY
BANK PLC Dr. FRANK JOHN on his 24hour Direct Telephone 234-8034113481,FAX:234-1-7592983

Because of time deference and arrange on the mode of transferring of your
accrued interest payment approved by the Senate House and Federal Government.We
are paying you interest of US $ 3.5 million dollars.

We advise you to desist from having other contacts until your payment is
confirmed for your own interest reconfirm to him your banking information's.And
send your telephone and fax number to him Thank you for your anticipated
cooperation and remain bless.

Thanks for working with Federal Republic of Nigeria

President, Federal Republic of Nigeria


                               SENATOR JIM NWOBODO

----- End forwarded message -----
The fact that Nigeria has a House and Senate like the US sounded suspicious, since few other countries do, so I checked the president's name and government structure against the CIA World Factbook http://www.cia.gov/cia/publications/factbook/geos/ni.html
They seem to be correct, surprise, although of course maybe the spammer himself read the Factbook entry and based his letter on it.
Now, why is the President of Nigeria sending an official e-mail from a Brazilian address and asking people to contact an agent in Italy. Does Nigeria not have a working e-mail system? Oh, of course it does, that's where the previous 419 scams came from.
(!) [Jimmy] Pity I couldn't reply to this at the time. Oh well.
Basically, Ireland has a House of Representatives and a Senate, except they are know by their Irish names (Oireachteas and Seanad respectively) -- several countries have modeled their system of government on America's (in Ireland's case it's not surprising as DeV, who framed the constitution, was American. Pity he decided to "improve" on the rights granted by the American Contitution though -- rather than free speech and freedom of the press, we're only free as long as the powers that be think we're being "decent" (porn was illegal here when I was a pre-teen -- even things like Playboy were against the law), and it took the pregnancy of a (IIRC) 13 year old rape victim for the constitution to be changed to even grant women the right to travel abroad for an abortion.
But at least we have the right to potable water and to defecate.
Oh... I think I've ranted about this before. Whoops.

(?) Hitchhiker's guide trailers

From Jason

The "Internet trailer" is hilarious. Avoid "lo-res" option if you can, the quality is horrible. Go with med-res or higher.
(!) [Brian] Note to self - enjoy high bandwidth while the living is good, you'll only be able to afford dialup when you're on what's left of Social Security.

(?) Well, that depends on a couple things. There's nothing left of Social Security right now. It takes money from young people, and gives it to old people. There's no money in the system. Oh, sure, there's a "trust fund", but as I understand it, nearly all of it is "loaned" to other government programs. That is, we've spend the money, and we've promised ourselves we're going to make up for it later. Given the spending habits of the government, how much you wanna bet we ever get around to it?

I don't think the Social Security program as we know it can continue for long. It's a bugger to deal with. On the one hand, I don't think the government should really be moving money around in that way. It's the responsibility of the individual.

But on the other hand, we've said "No worries, we'll always be there for you with this bit of money every month." We were, of course, lying for political expediency. It's obvious the whole thing is going under and was flawed from the start. But that doesn't matter now. We've got the program, whether we like it or not.

So, the question is, how do you faze it out while still being fair to everyone? The answer is: You can't. You can rip off seniors by cutting off "benefits" or you can rip off young people by having them pay in and receive nothing back. There's no nice way out now. The thing was a pyramid scheme from the start: Somebody's got to lose.

(!) [Jimmy] Heh. Let me say here that I love the PRSI system in Ireland (Pay Related Social Insurance) -- a tax that goes towards things like pensions, the dole, etc.
The thing I found to my advantage was that once you've paid into it for a certain amount of time, the government subsidises your medical & dental care -- I had to get a tooth pulled last month because I had an abcess (took nearly a whole month to drain completely too, dammit, but at least after Good Friday I could forget about it) and had to pay full price. When I went back to get some fillings done (yeah, it had been years since my last visit to the dentist) the dentist owed me money :)
(I love my dentist -- one of the funniest people I've ever met. My brother Joe had to get a tooth pulled: "Now Joseph -- I hope you don't mind my calling you Joseph instead of Joe, since I will be putting pliers and needles in your mouth -- you may feel a slight twinge of pain as I ram my knee into your chest and rip the tooth from your head.")
(!) [Brian] That said, I pulled down both trailers, about a minute and a half each to get the ~52G files down.

(?) That would probably be megabytes, not gigs.

(!) [Brian] Indeed. That was a fingerfault induced by giggling. So sorry.

(?) Either way, I envy your connection.

(!) [Jimmy] I envy any connection right now :)
(!) [Brian] OB Linux: Xine is a great goodness. I watched the trailers.
I laughed, I cried, I realized that no one was going to get an Oscar out of this and that indeed I might enjoy it very, very much. Mos Def appears to make a better Ford than I had suspected, and possible perfect casting has been executed to get Martin Freeman as A. Dent, and Alan Rickman as the voice of Marvin.
As with everything that DNA did, there appear bits here that aren't in any of the previous versions of the same work (aka Radio vs. Books vs. TV), so I'm not surprised that I see things that I don't recognize.
Yes, I think this is going to be an opening day event for me.
(!) [Jimmy] Well, H2G2 has been released, and all in all I think it was excellent (though the Arthur/Trillian thing did get hit over the head with a Hollywood hammer a but too hard).

(?) 419 spam

From Sluggo


That should be easy enough to verify.


"The" UK e-mail database. Is that like the phone directory? But it includes non-UK addresses too? Did you steal it from MI5? What "solid information" did the directory contain that led you to choose me as a partner?

(!) [Thomas] I'd imagine that it would be MI6 in this case -- I never did see the need for two separate groups, they all do much of a muchness anyway [1].



The Deputy Manager of Chevron's British division doesn't know how to spell "until"? And his keyboard doesn't have lowercase letters? Did they give you a cheap keyboard? Or did your son wire up an Apple ][ or TRS-80 keyboard to your computer? Or did you just spill coffee on your Caps Lock key?


And you will do the same?


Now you're the Managing Director for all of Chevron? Pleased to meet you, I'm a shareholder. I'll see if you're listed in the annual report....

Did you forget the Texaco part of the name? Chevron is now ChevronTexaco. Yes, I know your secretary would have corrected it, but you didn't want them to see this SECRET AND CONFIDENTIAL document.


Just a hint. Since you used capital letters throughout, I almost didn't realize we're talking about FIVE MILLION FIVE HUNDRED THOUSAND GREAT BRITIAN POUNDS (L5.500.000). I'm glad I read this e-mail carefully before deleting it. Sorry, my keyboard doesn't have a pound symbol, but yours doesn't either.


And what are the Englsih people going to do without petrol? You're so selfish. Is Englsih pronounced "Ingle-zee"? Where is that located? Is it one of the islands off Australia?


No, you're an Ingle-zee I guess.


Yes, London is so much more segregated than Los Angeles.


Take that, Queen Elizabeth II!

< Heh heh, he said 'cover up'! >
/\/(       /(__)
   | W----|| |~|
   ||     || |~|  ~~
             |~|  ~
             |_| o

[That's "cowsay -f bong", a cow with a bong.]


You know, I'm the great grandson Lloyd Tevis, the guy who founded Pacific Coast Oil that eventually became ChevronTexaco. I don't like the way you're embezzling from grandpaw's company. I just slapped you with a triple-damage lawsuit for $30,000,000.15 (THIRTY MILLION UNITED STATES DOLLARS AND FIFTEEN CENTS).

> francisinchevron@myway.com
> or FAX +44 870 130 79747).

Actually, time did run out. I got an international warrant to prevent you from leaving Britian until this is resolved. Scotland Yard was especially interested in your misuse of the UK E-mail database. So you're going to be staying in England for a long time, sucker. Here's a hint: try the curry and kebabs, they're really good. Skip the Yorkshire pudding, it's overrated.

(!) [Thomas] Pah! Whatdaya mean the Yorkshire pudding is overrated? :)

(?) When you're used to "pudding" as a smooth silky sweet dessert, and then you're all excited coz there's pudding on the menu, and then you get it and it's this bread thing, it's a real disappointment. That happened when I was seven and I've been traumatized ever since.

(!) [Thomas] Hahahahaha. Sorry, I shouldn't laugh. Hehehehehehehe. Ordinarily, a "pudding" is a sweet-dish, yes. There's also "steak and kidney pudding" -- the "pudding" part made from suet [1]. But that's savoury. You should try and look at a Yorkshire pudding in a new light -- cumberland sausages [2] in a huge yorkshire pudding with onion gravy, is divine. Mmmmmmm. It's a typical English dish, for sure.
(!) [Brian] Just be glad it wasn't black pudding, or you'd still be "7" in your head, and in a rubber room someplace...
(!) [Thomas] It's very nice cold with lots of strawberry jam. Hehehe, if you think England can be summed up just from Yorkshire puddings, kebabs and curry, you need to come back here, Mike... :)

(?) Yeah, I still need to take the Thomas Adam south coast tour.

(!) [Thomas] Hehehe, any time, Mike, any time. "Look, there's a rock in the sea, there's another one." You'd probably like Lulworth Cove and Durdle Door... (google for them, and you'll see. From a geological aspect they're world famous, and as small as the two places are, they attract a lot of people each year om holidays. I should know, I used to work right on the coast of Lulworth... :))
[1] For those of you who don't know, suet is just rolls of beef fat,
essentially that cook out to form some delicious dishes -- often it goes to make dumplings (with stew), or jam rolly-polly, yay! (That's a sweet dish), a pudding, if you will... :)
[2] Cumberland used to be a county bordering the Scottish border, along
with Northumberlandshire. The two merged to form "Cumbria" as we know it today.

(?) The white cliffs of Dover were pretty cool. Didier and I saw them during my last trip. Speaking of Didier, ou^ est cet garc,on? A-t-il quitte' la Bande de Re'pondants?

(!) [Didier] Mea culpa: it's true I haven't been answer-ganging for quite a while now...
(!) [Ben] Hey, Didier! It's good to see you here again! I've thought of pinging you once in a while just to see how you're doing.
(!) [Didier] Ah, thanks! It's very kind of you, Ben. As you can see, everything's fine here.
(!) [Didier] As excuses, many changes in my situation, the most crucial one being that this old friend of mine turned out to be the woman of my life (BTW: congratulations, Ben!);
(!) [Thomas] Yay! Congratulations. It's odd people are mentioning general awakenings and marriages... quite a few of the people I went to school with are now engaged! They're only the same age as me -- 21, I think it far too young yet, but...
(!) [Didier] Yeah, that's the way things go most of the time. I've seen this quite a lot... over the years. But hey, I'm somewhat older now. :)
(!) [Thomas] Congratulations, Didier. :)
(!) [Didier] Thanks, Thomas!
(!) [Ben] [grin] Yeah, I know what you mean. That's all right; my honey is quite geeky as well, and we both have our "must spend time with my digital spouse" moments.
(!) [Jimmy] Did I send my congratulations? If not, erm... congratulations.
(!) [Ben] Thanks, Jimmy! We're looking at your email here side by side, both of us grinning and grateful for your congrats.
(!) [Jimmy] Glad to hear it. (Or read it... YKWIM)
(!) [Didier] :) Not that I'm completely offline, but let's just say I now make a much more reasonable use of my time. Before I met her, I often used to forget what is usually considered as prime necessity -- you know, those things which prevent one from starving to death or suddenly falling asleep while crossing a street at peak hours... :)
What's extremely cool is that I convinced her of how fun, for instance, using the Gimp can be (and not the win32 version). There's still some work to do with her brother, though, but he already plans to try a French version of Knoppix, Kaella, as soon as he finds the time.
(!) [Didier] so I spend much less time these days in front of my Linux boxen -- although I found the time quite recently to get a decent mail server setup on my Debian box (some nicely tweaked MimeDefang milter, with hooks to clamav and SpamAssassin, which made my TAG account usable again).
(!) [Thomas] That's how it should be, of couse. :)
(!) [Didier] Indeed, and it (almost) works like a charm now.
(!) [Ben] Excellent!
(!) [Didier] Oh, not quite (yet :). There's still the need for some minor tweaks... There's so much broken mailing-list software out there. Even when their mail bodies are not HTML cr*p, you can be sure to find non-rfc2047-compliant headers, containing a bunch of raw outl00k-friendly, everything-else-fiendly 8-bit characters. :(
(!) [Didier] Also had the opportunity to become a Linux teacher for newbies -- though unfortunately, not full-time yet.
(!) [Thomas] Ah, well done. :) That must feel rewarding to do that.
(!) [Didier] Very much. I still have some experience to acquire, however, to make it even more fun.
(!) [Didier] But I'm still here, alive and well!
Hey Thomas, next time I cross the Channel... Hopefully you'll be there!
(!) [Thomas] Mais oui, bien sur! I hope that to be the case, also. I realy ought to look at visiting France again.
(!) [Didier] Then make sure you make a stop in Belgium and tell me!
(!) [Thomas] The last time I was there (which was also my first time) was ten years ago now. Mmmm, I can almost taste the food. I hope the onion soup is still as nice. :)
(!) [Didier] You really liked onion soup at 11? :) It took me many more years to come to appreciate it. But rest assured: it is.
(!) [Didier] Oh, and speaking of culinary oddities, my gf and I have just tried this "receipt of kitchen" (sorry, I couldn't help myself :)
(!) [Thomas] :D I want more of this -- a TAG Recipe column. Sod 2cent-tips. It's food recipes the readers want (this is something even procmail would like :P).
(!) [Didier] How about this? ;)
* ^Subject:\/.*(recette|cuisine|gastronomie)
        TWO_CENT_TIP=`echo "$MATCH" | translate --from=fr --to=en`
        :0 hfw
        | formail -I "Subject:$TWO_CENT_TIP"

	:0 bfw
	| translate --from=fr --to=en

	! tag@lists.linuxgazette.net
=================== Example 'translate' script =======================


OPTS=`getopt -o "f:t:" --long "from:,to:" -n "translate" -- "$@"`
test $? -eq 0 || exit 1

eval set -- "$OPTS"

while true; do
   case $1 in
   -f|--from) FROM=$2; shift 2;;
   -t|--to)   TO=$2;   shift 2;;
   --)        shift; break;;
   *)         exit 1;;

RECIPE=`while read line; do echo -n "$line " | sed -e 's/\&//g'; done`

wget -q -O- --user-agent 'Dummy HTTP client' --post-data='text='"$RECIPE"'&langpair='"$FROM"'|'"$TO"'&hl=en&ie=UTF8&oe=UTF8' http://translate.google.com/translate_t | perl -wne 'if (/<textarea/) {
        s!.*?<textarea.*?>(.*?)</textarea>.*!$1!; print "$_\n";
=================== Example 'translate' script =======================
(!) [Didier]


 Pouding de pain (mind the spelling)

 /Do not believe this word is from an English origin, it's actually
  French. The British adopted it one day and gave it its current
  spelling, 'pudding'.
  This bread pouding, similar to some other European recipes, was once a
  tasteful way not to lose remains of stale bread; it often used to feed
  little hungers at tea-time.
  At the time when the bread pouding was a family habit, the stale bread
  remains that were used were from good, old-fashioned bread; pouding
  made of modern bread is very likely to give disappointing results,
  unless one chooses so-called artisanal "country" bread./

   Raisins: 100g                            Fresh eggs: 4
   Amber rhum: 100cc                        Salt (optional): 1 pinch
   Stale country bread: 500g                Butter: 25g (for the mold)
   Cinnamon: according to taste             Ice sugar: plenty
   Sugar: 100g                              Milk: 250cc

 - Rinse the raisins quickly, remove the possibly remaining little
   tails; put the raisins in a bowl, add the rhum and let soak.
 - Break the stale bread into pieces. Put these in a salad bowl,
   sprinkle them with the milk until they fully absorb it; let soak.
 - Preheat the oven (180°C).
 - Dry the bread with one's fingers to remove the excess of milk; crush
   it with a fork. One should obtain evenly sized chunks, not too big
   but not a pulp either. Add the desired quantity of cinnamon, the
   sugar, the beaten eggs. Taste and correct with the salt if
   needed. Add the soaked raisins, as well as their excess juice.
 - Pour into a buttered mold, flatten the surface. Put in the oven.
   Cook for 30 minutes (use this time to re-read a few chapters from
   http://www.openldap.org/doc/admin22/guide.html, for example). Check
   with a knife: the blade must remain dry when the cooking is over.


Quick, easy, economical... and excellent.
(!) [Thomas] I've bought the ingrediants, and am intending to make this later. I shall let you know how it goes. :)
(!) [Didier] Bon appétit!
(!) [Ben]
Didier wrote:
>   /Do not believe this word is from an English origin, it's actually
>   French. The British adopted it one day and gave it its current
>   spelling, 'pudding'.
Hmm. My fiance?, who is quite food-geeky as well, will have quite a bit to say about that... but she wants to subscribe to TAG and explain it herself. Look out for a longish pudding-relevant post. :)
(!) [Didier] /me is waiting impatiently while his stomach growls. :)
(!) [Thomas] Good to have you back, Didier. Don't be a stranger to TAG anymore... :)
(!) [Didier] Thanks a lot, Thomas. I'll stay tuned.
(!) [Thomas] Oh, I was so proud of my purchase of what I had for dinner last night, I photographed it [2]. :D
[1] No, that's not based on any Shakespeare... [2] http://edulinux.homeunix.org/~n6tadam/sea_bream.jpg


> TEL/FAX: 44 870 130 79756.
> PRIVATE EMAIL:francisinchevron@myway.com

http://www.sfgate.com/cgi-bin/article.cgi?file=/chronicle/archive/2004/10/20/BUGPQ9CAGD22.DTL&type=business History of ChevronTexaco

(!) [John] Ha!
Aw, come on, Mike. The guy is just asking for help... and is offering to make the un^H^Hlucky sucker^H^H^H benefactor rich for the trouble. Just try to overlook his poor English grammer and spelling. Maybe he's one of those self-made executive types who made it in spite of being a 6th grade drop-out.
Whee! Now I can realize my (American) dream of retirement in a NY 5th Ave penthouse next door to Mr BG himself!

(?) Re: Pouding

From Ben Okopnik

Forwarding Kat's reply to Didier's "pudding" post. Hopefully, he hasn't starved to death waiting for it...
----- Forwarded message -----

pudding Look up pudding at Dictionary.com


pudding Look up pudding at Dictionary.com

c.1305, "a kind of sausage: the stomach or one of the entrails of a pig,

sheep, etc., stuffed with minced meat, suet, seasoning, boiled and kept till needed," perhaps from a W.Gmc. stem *pud- "to swell" (cf. O.E. puduc "a wen," Westphalian dial. puddek "lump, pudding," Low Ger. pudde-wurst "black pudding," Eng. dial. pod "belly," also cf. pudgy). Other possibility is that it is from O.Fr. boudin "sausage," from V.L. *botellinus, from L. botellus "sausage" (change of Fr. b- to Eng. p- presents difficulties, but cf. purse). The modern sense had emerged by 1670, from extension to other foods boiled or steamed in a bag or sack. Ger. pudding, Fr. pouding, Swed. pudding, Ir. putog are from Eng. Puddinghead "amiable stupid person" is attested from 1851.


Note the last bit: "Ger. pudding, Fr. pouding, Swed. pudding, Ir. putog are from Eng."
I don't believe that the etymology of "pouding" can be traced back in French further than the English, although I'm willing to be caught out on this, as medieval French is not my bailiwick.
One can trace a possible "boudin --> pouding" explanation, but "boudin" definitely means sausage in French, while "pouding" is a rice pudding/bread pudding concoction. In English "pudding" applies across the board from "black pudding" (clearly boudin noir) to the generic (non-American) "pudding == dessert in general unless otherwise modified" usage.
The English usage of "pudding" to mean dessert makes sense in that the original versions were all boiled puddings, whether savory or sweet.
Are the French not using "pain perdu" as a term anymore? This is the term used in English language cookbooks for what's known in the U.S. as "bread pudding", and previously as "poor knights". There is a version known as "bread and butter pudding" as well, which is similar.

----- End forwarded message -----

(?) Investment Relationship /waiting to hear from you.

From Eward Jr

Investment Relationship
Dear Respectful One,
This is not a spam email bellow are the following proposals with its full details with a sincere hope to develop some feasible business for our mutual benefits through your keen cooperation. Partnerships comprise two or more business partners pooling their resources in a business with a view to profit,therefore I would like to apply through this medium for your co-operation to secure an opportunity to invest and do joint business with you in your country. I have Inherited a substantial capital I honourably intend to invest in your country into a very lucrative business venture of which you are to advise and execute the said venture over there for the mutual benefits of both of us. Your able co-operation is to become my business partner in your country and create ideas on how money will be invested,properly managed and the type of investment after the money is transferred to your custody with your help and assistance. Mean while,on indication of your willingness to handle this transaction sincerely by protecting me and my family interests and upon your acceptance of this proposal,I would furnish you with the full detailed information,procedure,amount involve and mutually agree on your percentage interest or share holding for helping me to secure the release of the deposit and investing the money in your country under your proper management and care. I shall be glad to reserve this respect and opportunity for you,if you so desire,but do urge you to give the matter your immediate attention it deserves.If this proposal is acceptable by you,please do not make undue advantage of the trust i bestow on you,and your urgent call and reply is highly needed,for more detailed informations and oral talks. Looking forward to your candid and urgent call and positive reply today and a mutual healthy business relationship with you.
My Very Best Regards, Edward Jr. Email:ed1956r@yahoo.fr
(!) [Sluggo] A spam bellow or spam bellows?
/ If only I were an artist I could draw \
\ something besides cows.               /
       \    ____
        \  /    \
          | ^__^ |
          | (oo) |______
          | (__) |      )\/\
           \____/|----w |
                ||     ||


(?) Software patents

From Sluggo

For Jimmy and Mick:
http://www.nosoftwarepatents.com/phpBB2/viewtopic.php?t=407 Ireland in the doghouse over role in software patents (from Slashdot)
(!) [Barry] And me! Not that I want to be waving my "I am Irish" flag on this issue.
But, FYI, there is a source of local information and inititives
(!) [John] Well, brace yourselves. They've finally succeeded in ramming through the software patents directive in the EU, despite wide opposition in the European Parliment. Democracy is alive and well in the world (not!).
Get ready for ensuing battle.
(!) [Pete] It's still not fully passed yet. The commission have approved it, but it still needs to go back to the European Parliament for a second reading. However, this time it will take a 2/3 majority to /prevent/ it being implemented.
(!) [John] I expect that it will breeze through. If it didn't get killed today, it won't be killed. The attack on FOSS is about to begin in earnest.
(!) [Pete] No, I'm not yet convinced that this will pass - perhaps I'm being a little too optimistic, but there are a few governments with back bone in the EU who are still opposed to this (mine not being one of them unfortunately).
(!) [John] Indeed, if it were not for those few, namely Poland, Denmark, with support from Hungary, Latvia, the Netherlands, and Cyprus, it would have been rammed through at least 3 or 4 months ago.

(?) Sounds like the sports stadium syndrome. The team owners (billionaires) want the public to pay for a new stadium. The public votes it down -- twice -- but the city or state rams it through anyway citing "emergency economic necessity". This happened twice in my state, and now a third team is saying, "Hey, you did it for the other two, now it's my turn."

(!) [Pete] It would appear that today's events were as a result of 'process'. From what I've read the Danish minister could have lodged a request that the item be removed from the A list, instead of asking for it to be converted to the B list. Had he done the former the Luxembourg Council Presidency would have had to agree, but because he chose the latter they could decline the request (and it has also been suggested that this was his intent, having been forced by a Danish Parliamentary Committee to try and have it removed).
Anyone who lives in a European country and cares about the software patents issue should now be writing to their MEP explaining why they should be voting NO for this when it comes before the EU Parliament (which is likely to be sooner rather than later).
(!) [John] I only hope that you're right and I'm wrong. I know that there is considerable discontent among the MEP for the way that their votes have been trampled, but I'm not sure that's enough. Maybe they'll get thoroughly disgusted and react effectively.
Some say that the existance of the EP serves more to provide the illusion of a democratic process and that their vote is not really meant to be deterministic. Again, I hope it proves not be the case, but the whole matter has made a mockery of the concept of democratic process.

(?) Could somebody explain to us yanks what this means? Is the European Parliament similar to the British and Canadian parliaments? If they don't have ultimate power ("deterministic votes"), who does? Or is it just a case of corporations buying certain key representatives?

(!) [Thomas] EU law. Whole books have been written on the subject, and yet no one really understands it. My own idea of it, is that because we are a member of the EU, we have MEPs (which represent us in the EU). EU law rules over any specific laws each respective country might have. So in that way, it can be fairly dangerous... :)
(!) [Pete] As I understand it, there are two main institutions, the European Commission, which is made up of unelected officials from the various member nations, and the European Parliament, which is made up of representatives voted for by 27% percent or so of EU citizens, those who could be bothered to express a preference.
However, the Commission seems to have more power than the Parliament, a problem that the European Constitution is supposed to go some way in rectifying.
But as you say, whole books have, and will continue to be, written on the subject. :)
(!) [John] For a pretty good explanation, see
(!) [Barry] All is not lost yet:
From the office of Proinsias de Rossa (an Irish MEP):


Thanks for your message. Following yesterday's decision in the Competitiveness Council, the EP has requested that the Commission make a statement to the EP Plenary tomorrow in Strasbourg at 6.00pm. MEPs are awaiting with considerable interest to see what the Commission has to say but it's important to remember that the EP has the right to retable any of its first reading amendments not taken up in the Council's common position, which the EP has yet to see, or to reject the common position in its entirity. If Council still refuses to accept them, a 'conciliation committee' of MEPs and national civil servants will meet to reach a compromise. If none is found, the proposal falls. The important point is that the Ministers do not take the final decision on this issue - they must work with the EP.
We'll keep you posted.


Tomorrow is now today - by the by.

(?) Register news

From Sluggo

The Register confirms that cows can bear grudges and be lesbians, and "even chickens might have to be treated as individuals with needs and problems".
(!) [Pete] Shouldn't that be ...
< cows can bear grudges and be lesbians >
        \   ^__^
         \  (@@)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

(?) Dave Allen (was Re: [TAG] UPS Problems..)

From Neil Youngman

On Friday 02 May 2003 19:03,
(!) [Ben] Thomas has a sense of humor; why do you think he was defending Lucas?
:) It's a very subtle brand of humor.
(!) [Thomas] LOL, indeed my humour is subtle. I have a very dry sense of humour perhaps bordering on sarcasm on extreme occassions..... :)
(!) [Ben] <grin> I've always appreciated that sort. There was some English comedian... damn, can't remember his name, but he would sit in a spotlighted chair on the stage, in front of the curtain, and deliver these devastating lines with just a slight smile. Dry as a good martini (not that I think there is such a thing as a good martini...) - I was green with envy.
(!) [Neil] Sitting in front of a curtain in a spotlighted chair makes me think of Dave Allen, but he's Irish not English. If he had a glass of whiskey as well and his sketches were as lame as his standup (sitdown) comedy was sublime, then you're thinking of the same guy.
(!) [Ben] I'm thinking... that sounds right. The name certainly rings a bell. I can believe that I've made a mistake at this long remove - it's been years.
(!) [Neil] Sadly Dave Allen died this week aged 68.
(!) [Ben] Awww. He'll definitely be missed; that classic sense of humor he had was rare in today's world. Requiescat in pace, Dave.

(?) Cron Jobs not being run

From Suramya

15 * * * * root /etc/awstats/stats_update
(!) [Thomas] This is a bad idea. As I have pointed out in the past to people, having the time exactly on the hour (or intervals such that it is run on the hour) can cause clock skews, if one controls their time via ntpd, say. The result is such that the cron job may not run at all.
(!) [Brian] I see a cron entry set to run at 15 minutes after each hour. What do you see, Thomas?
(!) [Sluggo] I suspect it's a case of British clocks running in the opposite direction (do they go leftwards like cars do?), or maybe the spelling is different. Try "been" instead of "bin" in the script.
<Running, ducking>

(?) Question about file mtime on linux

From Sluggo

I smell an LG article coming....

(!) [Thomas] Not more bullshit, surely...? :P

(?) And what's this about article rubbish, Thomas? Do I have to slap you upside the head?

(!) [Thomas] Heh. That's just my warped sense of "\"humour\"". :)
(!) [Ben]
perl -we'
use ThomasAdam;
$ref=ThomasAdam -> new();
print $ref -> can( "ThomasAdam::humor" )
Use of uninitialized value in print at -e line 1.
Hmm. Let's try again -
print $ref -> can( "ThomasAdam::humour" )
Ah! The "ThomasAdam" object only has the British method...
(I'm teaching Perl this week. Can anyone tell? :)


From Ben

On Wed, Apr 13, 2005 at 09:55:02PM -0400, Jay R. Ashworth wrote:

> IANAKH, but I *believe* the pertinent code is in kernel/fs/$FILESYS/file.c

Sounds like a Hebrew title to me. Not quite as good as rabbi, but almost equivalent to a doctor...

(!) [Jay] Hey! Get away from me with that... what that thing a mohel uses?

(?) I dunno, a peepeechappa? :)

(!) [Jay] Do I have that right, Ben? :-)

(?) An Ianakh like yourself would know better than humble little me. :) You lost me right after "The specific answer"; I got through the next five words via simple inertia, but then my brain seized up and had to be resupplied with copious amounts of good food. Unfortunately, I'm still waiting for the delivery boy (yum, /bulgogi/ with /gochujang/ and an armada of little side dishes, kimchi on the side. Woohoo!)

(!) [Jay] My turn for "silly me".
Actually, I think you may just be wimping out because you don't wanna delay dinner... :-)

(?) You're right that I didn't want to delay dinner (just as yummy as I'd expected. Mmmm...), but I also honestly had no idea. My little bits of spelunking in the kernel source have been aimed elsewhere.

(?) PAM auth failure - can't login in.

From Ben

Good luck! Oh, and check out the "Asking Questions of The Answer Gang" FAQ, at (http://linuxgazette.net/tag/ask-the-gang.html); there's a lot of helpful info there, including [cough] a request to avoid top-posting.

:) We're always glad to help people with their Linux problems, but the

emphasis is on "people" - these conversations are published in LG so they may help more than just the person asking the question - and it gets to be rather painful for the people doing the formatting if standard email etiquette isn't followed.

(!) [Sluggo] What's happening around here? "Gentle" Thomas has been chewing people out for top-posting, and now "007" Ben is ultra-polite. I guess your birthday present is a pair of dark sunglasses, Thomas.

(?) You've mistaken "sweet-talking while reloading the BFG10K" for politeness. Tisk, tisk, tisk.

*** CHH-CLACK ***

Ahhh. Much better now. Bring on the Stroggs!

(?) My PC is slower than yours...

From Ben

A Linux kernel compile is considered to be one of the best memory tests in the world; I've had it find bad memory on machines that ran Wind0ws for years without any problems.

In fact, I find Linux to be generally more hardware-intensive than Wind0ws.

(!) [John] Perhaps one could say that Linux makes better use of the potential of the hardware!?

(?) Yeah, that's probably an even better way to phrase it. This is one of the reasons that you an still accomplish useful tasks - hell, pretty much everything you'd expect out of a computer - on a 386 with 8MB RAM and a 20MB HD. Try doing that with Wind0ws...

It's true that most of us prefer newer, faster machines. That's not the point; it's that people who don't have the latest and greatest can still have access to computing resources. That is awesome.

(!) [Sluggo] In other words, Linux is scalable.
I'm not sure 386/8/20 is still sustainable though. The kernel size has been creeping up, although how much of that is essential and how much is just people needing more features nowadays, I'm not sure. I did run Linux on some 386 routers in the mid 90s, using Slackware.

(?) Just read a "report from the field" yesterday that reports it still to be so. Heck, considering that a lot of floppy-based Linux routers run just fine without a hard drive at all...

(!) [Jay] Not to mention our SnapGear monolithic routers, which run out of 4MB of execute-in-place flash and 4MB of RAM quite happily.

(?) Sounds similar to "the world's smallest server" that they had for a while at http://wearables.stanford.edu (sadly defunct now, it seems.) They were running RH Linux on a 5MB flash chip, IIRC.

(!) [Jay] Ok, that's a special case; the Coldfire is a nice chip.
But has anyone here seen the Linux-box-on-an-RJ45-*socket*?

(?) Wow. Where's this gadget?

(!) [John] There was mention of it on /. a coupla weeks ago ...
(!) [Jay] PicoTux:

(?) Oh, yeah. Total geek fun. Thanks!

(!) [Thomas] I still run Linux on my 386. I change distros on it quite often. Currently it is running SuSE 6.4. It's my Rsync server. :) [1]
[1] That's right -- nothing important to backup.
(!) [John] It's a stretch even with W2k on a 1.2Ghz P-III! It's (almost!) incredible just how miserable the "multi-tasking" is on any M$ os. Can't really say too much about XP, as I do all I can to avoid it, but I suspect it's really no better.

(?) off-topic

From Heather

spam all month has been telling us that geeks are the best stock picks. ;P

/me avoids the flying brickbats

ok ok back to drawing the cover art :) Konqi will be celebrating the wearin' o' the green...

(!) [Sluggo] I like the ones about TAG's eBay and PayPal accounts.

(?) need information about sylpheed-claws

From Karl-Heinz

The default setup also has a size limit above which the mail is not checked to avoid spamassassin beeing bogged down by huge mails.

(!) [Ben] Heh, that's a non-solution to a real problem. What are you supposed to do if you do get huge mails? Say, your boss decides to send you a two-line memo in MS Word - that's got to be a couple of gig, right? :)
The anti-virus people have been jockeying with this kind of thing for a while. The problem is, of course, that compressed files could contain a virus, so you'd have to decompress them to check for one...
ben@Fenrir:/tmp$ dd if=/dev/zero bs=1GB count=1|bzip2 > foo.bz2
ben@Fenrir:/tmp$ ls foo*
-rw-r--r--  1 ben ben 722 2005-07-07 21:05 foo.bz2
ben@Fenrir:/tmp$ mutt -s 'Hi, Karl-Heinz!' -a foo.bz2 kh@somewhere.com < /dev/null
(!) [Jason] Perhaps apropos of the swap space discussion elsewhere, I tried your command, and...
~$ dd if=/dev/zero bs=1GB count=1|bzip2 > foo.bz2
dd: memory exhausted
~$ dd --version
dd (coreutils) 5.2.1
Written by Paul Rubin, David MacKenzie, and Stuart Kemp.

Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
I have 128 megs of RAM, plus 150 or so of swap. Looks like dd tried to grab the whole thing at once. You either have a different version of dd or a much nicer setup than me. So for people without that much VM kicking around, you could use:
~$ dd if=/dev/zero bs=1M count=1024 | bzip2 > foo.bz2
1024+0 records in
1024+0 records out
(!) [Ben] Heh. Not necessarily nicer, but I tend to do some large-scale stuff once in a while, so I tend to prefer to configure it by using a Mack truck's front bumper as a model.
ben@Fenrir:~$ free -tk
             total       used       free     shared    buffers     cached
Mem:        515464     320600     194864          0      23032     117328
-/+ buffers/cache:     180240     335224
Swap:      1951888          0    1951888
Total:     2467352     320600    2146752
In other words, 512MB of RAM and a couple of GB of swap (none of which is being used at the moment, but I've just rebooted after configuring a shiny new kernel.)
(!) [Jason] You'd have to use "bs=1MB count=1000" to make the size match Ben's command.
(!) [Ben] Right on both counts. I wasn't expecting people to actually do this, just to become aware of it. It is an interesting sort of thing to play with, though - 1GB shrunk to 700-some _bytes?_ Wow.
(!) [Jay] Depends on what you're compressing.
We use Microlite's BackupEdge on our Linux/filePro systems, and it's not uncommon for us to see 70-80% compression on the key (data) segment files, and I've seen over 90% on some.
(!) [Ben] This was %99.99993 compression, Jay. Don't know about anyone else, but it impressed me.
(!) [Sluggo] There was a book I read when I was little about a boy who met a Martian who had a spaceship disguised as a car. It was the intersection of two technologies: (1) the spaceship walls couldn't be seen except with special glasses, and (2) you could fold the car up and put it in your pocket. Now that's compressibilty. I've often thought about how that would solve the problem of parking spaces....
"Special glasses", heh heh. Now we know Ben's a Martian. In addition to being a secret agent, computer consultant, and ship captian. "Yoga instructor", right, likely story.
(!) [Ben] Actually, in my mid-teen years, I "invented" an inflatable car that would go a long way toward that scenario. Seriously. Did the research, found out what it would take... looking back at it today, it's still quite a reasonable proposition. It was an ACV, of course; tapped the engine output to inflate and deflate (via a tiny two-chamber extraction pump) the passenger compartment. You could stack these things, deflated, on top of each other, in your garage or whatever. Steered via a joystick, way safer than a regular case in case of an accident... very simple design, but - alas, I've never had the cred in that world or the money/desire to do it myself. Oh well.
(!) [Jay] And it acts like bubble-wrap if you get in a crash. Neat.
(!) [Ben] Hopefully without the popping sound :) - but yes, that was the idea. I actually foresaw teenagers hotwiring the family car and playing bumper-cars/wreck rally... us humans mis-use everything to its highest capacity, but that one didn't seem to be nearly as bad as doing the same with the metal boxes we've got now. Given that the energy of impact (i.e., kinetic energy, KE=1/2mV2) is very strongly dependent on deceleration time - which, /ceteris paribus/, is strongly dependent on the amount of 'give' in the colliding bodies - these cars would be much safer than pretty much any other car on the road today.
(!) [Ben] Now, yeah, you could have a compressor/decompressor that works like this:
("Compressed file" viewed with 'bvi'; last position is the value of the repeated byte.)
00000000  33 62 39 61 63 61 30 30 00                      3b9aca00.
# "Decompressor"
perl -wpe's/(.*)(.)$/$2x$1/e' file>file.out
Then you'd get %99.9999991 compression with a 1GB file, and %99.9999999991 with a 1TB file. Would that impress you? :)
(!) [Jay] There's a lot of air in the average database file.
(!) [Ben] Oh yeah. I recall getting the kind of compression ratios you're talking about when I was DBAing in St. Thomas; made me wonder, until I actually looked at those files (with Norton's DiskEdit, if I recall correctly.) Lots of air.
(!) [Ben] If you project that kind of compression ratio to, say, 1TB or so, you can see how it might fry an AV utility's little brain... and if Mr. Cracker is lucky, it might totally fill up the partition as well.
(!) [Jason] That's just evil, Ben. I wish I'd thought of it.
(!) [Ben] [grin] Me too. I was just riffing off something that a poster on Bugtraq said a while back; he didn't approach the problem exactly this way, but it was implicit in his idea. A clever fellow, that one.
(!) [Jason] I played around with this a bit more, using multiple passes with both gzip and bzip2. Here's what they produced with a gig (230 bytes) of zeros:
~/prog/compression$ ls -l
total 1044
-rw-r--r--  1 jason users     785 Apr 14 15:12 zero.bz2
-rw-r--r--  1 jason users     107 Apr 14 15:13 zero.bz2.bz2
-rw-r--r--  1 jason users     143 Apr 14 15:13 zero.bz2.bz2.bz2
-rw-r--r--  1 jason users 1042069 Apr 14 15:04 zero.gz
-rw-r--r--  1 jason users    2780 Apr 14 15:06 zero.gz.gz
-rw-r--r--  1 jason users     324 Apr 14 15:06 zero.gz.gz.gz
-rw-r--r--  1 jason users     361 Apr 14 15:06 zero.gz.gz.gz.gz
(!) [Ben] Oh, cool!
(!) [Jason] No special options were passed to either compressor, so this is the defaults. I'm currently compressing a terabyte of nothing with bzip2. This will probably take a couple days, so I'll report back my findings. As if anybody cares. :-)
(!) [Ben] Oh, I do! I'd really appreciate it if you'd send me a copy. Something about the idea of this very tightly-coiled 500-mile-long spring (ready to explode into a tangled mess of metal spaghetti if you poke it) quietly living on my system really appeals to me. :)
(!) [Jay] It's sort of deviantly hackish. Name it something that a nosy hacker will want to uncompress, and just plop it there.
(!) [Breen] Unless, of course, you manage to tempt him to uncompress it in situ.
(!) [Jason] Here's the sizes for terabyte (230 bytes, not a trillion bytes) bzip2'd:
~/prog/compression$ ls -l terabyte.*
-rw-r--r--  1 jason users 766574 Apr 17 23:10 terabyte.bz2
-rw-r--r--  1 jason users    238 Apr 18 09:12 terabyte.bz2.bz2
-rw-r--r--  1 jason users    324 Apr 18 09:12 terabyte.bz2.bz2.bz2
Again, it looks like two passes is the optimal for bzip2 when dealing with weird data like this. I didn't try gziping because the resulting file would be about a gig, but it occurs to me I could do two-pass compression on the fly:
~/prog/compression$ dd if=/dev/zero bs=1M count=1048576 | gzip | gzip > terabyte.gz.gz
...so I'll have some figures on gziped data soon. <grin>
(!) [Ben] [laugh] I'd be curious.
(!) [Jason] Anyway, since you request it, I'll send you a copy of it. terabyte.bz2.bz2 is so small I'm tempted to attach it to this email, but I'm afraid that someone on this list might have some sort of AV setup that would die on something like this.
(!) [Ben] Got it - thanks! It will live quite comfortably in my "zoo" (a directory full of stuff that I use to teach SC-300, Sun's security course.)

(?) I hope spamassassin doesn't choke on this. But I would like a copy too :-) I could rename it in "~/allmypasswords.bz2" or something like that.

(!) [Breen] Evil. Pure evil. May I have a copy too?
(!) [Jason] Ah, you guys are so lazy. All it takes is a couple days of CPU time. :-)
Anyway, I'm still afraid to attach it in the clear in case anybody has a badly configured spam/AV setup. Do you guys feel as if this fear is justified? Anyway, uuencode isn't cooperating, so we'll do it this way:
#include <stdio.h>
static char message[] =
int main(void) { size_t i; for(i = 0; i < sizeof(message) - 1; ++i) { putchar(message[i]); } return 0;}
Compile, and redirect output to "terabyte.bz2.bz2". Or whatever you want to name it. Just be aware it's compressed twice.

(?) That would indeed be a nasty mail bomb. On the other hand I felt rather save from viruses on a Dec alpha/64 bit running OSF Unix which was even hidden behind a firewall and did not receive mails directly.

(!) [Ben] ...which is, of course, The Right Thing in many cases.
                __ Mail server __
--> Router/FW <                   > Router/FW --> LAN
                -- Web server ---
Both of the servers, of course, have a minimal tool set (just enough to accomplish the job they need to do) so that Mr. Cracker has nothing to work with if he should get that far... so cracking the outside router AND root on the servers that have a public "face" leaves him essentially where he was before he started - or worse. Better yet, use Kapil's "transparent FS" idea: the reads go through to the FS, but the writes to everything except, e.g., the mail spool and the logs go off to /dev/null... let'em figure that one out. :)
(!) [Jay] Oh, that is absolutely devious.
I like it. :-)
(!) [Ben] [bows] Thannnk you; Kapil's got the wheels spinning in my head on this topic, although I have no time to fiddle with it for now. Sure, it's security through obscurity... but face it, how many crackers would even have the slightest clue? And what could they do about it if they did, given that the tools are either missing or are sitting on the far side of the chroot jail? It's kinda like using 'chattr' on certain key files and then taking 'chattr' with you when you go away - only with more hair, teeth, and muscles. :)

(?) Still for the querent I think the best is to to decouple the gui-mailer and the filtering to avoid long waits and hangs. Then spamd/spamc should do nicely even on a rather meager system. I did not get the impression that fetchmail was exceptionally memory hungry, just CPU spikes.

(!) [Ben] [Nod] That's my experience as well. It seems to me that anytime you can use an "older", console-based Unix utility or an older, time-tested daemon, you're better off in terms of resiliency and broader capabilities; certainly in regard to the range of things that can be handled without bogging the system.
I use GUIs the way some people make sushi: the stuff on top is just flavoring, not the actual food (the /gohan/ is what holds that distinction (http://www.japan-guide.com/e/e2043.html)). And I agree with Brian: a mouse is indeed a device for selecting which xterm you type in. Except I usually use the Alt-Tab key. :)
(!) [Jay] Naw; the mouse is for copying text between xterms.
(!) [Ben] Oh. Well, yeah. I wasn't saying that mice aren't useful; they're just not the linchpin without which the world of computers will grind to a screeching halt. Nice, but not critical, despite what many people think.

(?) Spam jokes

From Jimmy

The engineer's terms
Top 25 Engineer's Terms and Expressions (What they say versus what they mean)
A number of different approaches are being tried. (We are still guessing at this point.)
Close project coordination. (We sat down and had coffee together.)
An extensive report is being prepared on a fresh approach. (We just hired three punk kids out of school.)
Major technological breakthrough! (It works OK; but looks very hi-tech!)
Customer satisfaction is believed assured. (We are so far behind schedule, that the customer will take anything.)
Preliminary operational tests were inconclusive. (The darn thing blew up when we threw the switch.)
Test results were extremely gratifying! (Unbelievable, it actually worked!)
Casual day
Week 1 - Memo Number 1: Effective this week, the company is adopting Fridays as Casual Day. Employees are free to dress in the casual attire of their choice. Week 3 - Memo Number 2: Spandex and leather micro-miniskirts are not appropriate attire for Casual Day. Neither are string ties, rodeo belt buckles or moccasins. Week 6 - Memo Number 3: Casual Day refers to dress only, not attitude. When planning Friday's wardrobe, remember image is a key to our success. Week 8 - Memo Number 4: A seminar on how to dress for Casual Day will be held at 4 p.m. Friday in the cafeteria. A fashion show will follow. Attendance is mandatory. Week 9 - Memo Number 5: As an outgrowth of Friday's seminar, a 14-member Casual Day Task Force has been appointed to prepare guidelines for proper casual-day dress.
Job Interview Quotations
Vice Presidents and personnel directors of the one hundred largest corporations were asked to describe their most unusual experience interviewing prospective employees.
A job applicant challenged the interviewer to an arm wrestle.
Interviewee wore a Walkman, explaining that she could listen to the interviewer and the music at the same time.
Candidate fell and broke arm during interview.
Candidate announced she hadn't had lunch and proceeded to eat a hamburger and french fies in the interviewers office.
Candidate explained that her long-term goals was to replace the interviewer.
Candidate said he never finished high school because he was kidnapped and kept in a closet in Mexico.
Balding Candidate excused himself and returned to the office a few minutes later wearing a hairpiece.
Applicant said if he was hired he would demonstrate his loyalty by having the corporate logo tattooed on his forearm.
Applicant interrupted interview to phone her therapist for advice on how to answer specific interview questions.
Candidate brought large dog to interview.
Applicant refused to sit down and insisted on being interviewed standing up.
Candidate dozed off during interview.
First class
A blonde was headed to Detroit. She got on the plane and sat down in first class. A few minutes later, a flight attendent came up to her and told her that her ticket was for coach and she had to move from the seat. She refused. The flight attendent was persistant, but the blonde replied, "No, I want to sit here, I've always wanted to see what it is like in first class." The flight attendent was getting frustrated. Finally, after quite some time, she convinced her to move. Another passenger who had witnessed the exchange asked the attendent, "How did you get her to move?" The flight attendent replied, "I told her first class doesn't stop in Detroit."
John and Marsha decided that the only way to pull off a Sunday afternoon quickie with their ten-year-old son in the apartment was to send him out on the balcony and tell him to report on all the neighborhood activities. He began his commentary as his parents put their plan into operation. "There's a car being towed from the parking lot," he said. A few moments passed. "An ambulance just drove by." A few more moments passed. "Looks like the Andersons have company," he called out. "Matt's riding a new bike." "The Coopers are having sex." Mom and Dad shot up in bed. "How do you know they're having sex?" "Their kid is standing out on the balcony too."
A popular bar had a new robotic bartender installed. A fellow came in for a drink and the robot asked him, 'What's your IQ?' The man replied, '150.' So the robot proceeded to make conversation about Quantum physics, string theory, atomic chemistry, and so on. The man listened intently and thought, 'This is really cool.' The man decided to test the robot. He walked out the bar, turned around, and came back in for another drink. Again, the robot asked him, 'What's your IQ?' The man responded, '100.' So the robot started talking about football, baseball, and so on. The man thought to himself, 'Wow, this is amazing.' The man went out and came back in a third time. As before, the robot asked him, 'What's your IQ?' The man replied, '50.' The robot then said, 'So, you gonna vote for Bush again?'
Stud Tires Out
Panda Mating Fails; Veterinarian Takes Over
Soviet Virgin Lands Short of Goal Again
British Left Waffles on Falkland islands
Lung Cancer in Women Mushrooms
Eye Drops off Shelf
Teacher Strikes Idle Kids
Reagan Wins on Budget, But More Lies Ahead
Include your Children when Baking Cookies
Something Went Wrong in Jet Crash, Expert Says
Police Begin Campaign to Run Down Jaywalkers
Safety Experts Say School Bus Passengers Should Be Belted
Drunk Gets Nine Months in Violin Case
Survivor of Siamese Twins Joins Parents
Farmer Bill Dies in House
Iraqi Head Seeks Arms
Seeing the light
A 70-year-old man went to the doctor's for a physical. The doctor ran some tests and said to the man, "Well, everything seems to be in top condition physically, but what about mentally? How is your connection with God?" And the man answered, "Oh me and God? We have a really tight bond, he's so good to me. Every night when I have to get up to go to the bathroom, he turns on the light for me, and then, when I leave, he turns it back off." The Doctor was astonished. He called the man's wife and said, "I'd like to speak to you about your husband's connection with God. He claims that every night when he needs to use the restroom, God turns on the light for him and turns it off for him again when he leaves. Is this true?" And she said, "That idiot, he's been peeing in the refrigerator!"
All generalizations are false, including this one.
All generalizations are useless, including this one.
All good things must come to an end, I just want to know when they start!
All great discoveries are made by mistake.
All I ask is the chance to prove that money cannot make me happy.
All inanimate objects can move just enough to get in your way.
All our dreams can come true if we have the courage to pursue them. - Walt Disney
All probabilities are really 50%. Either a thing will happen or it won't.
All syllogisms have three parts; therefore this is not a syllogism.
All the world is a stage and most of us are desperately unrehearsed.
If Oracle made toasters... They'd claim their toaster was compatible with all brands and styles of bread, but when you got it home you'd discover the Bagel Engine was still in development, the Croissant Extension was three years away and that, indeed, the whole appliance was just blowing smoke.
If Hewlett-Packard made toasters... They would market the Reverse Toaster, which takes in toast and gives you regular bread.
If IBM made toasters... They would want one big toaster where people bring bread to be submitted for overnight toasting. IBM would claim a worldwide market for five, maybe six toasters.
If Xerox made toasters... You could toast one-sided or double-sided. Successive slices would get lighter and lighter. The toaster would jam your bread for you.
If Radio Shack made toasters... The staff would sell you a toaster, but not know anything about it. Or you could buy all the parts to build your own toaster.
If Thinking Machines made toasters... You would be able to toast 64,000 pieces of bread at the same time.

(?) Re: CD/RW's on Fedora Core 3

From Brian

The area of interest is in a technology called Mount Rainier.

(!) [Sluggo] Mount Rainier???

(?) Mount Rainier.





That last is my favorite, for the page title: "We put the F in packet writing."

The correction? Anyplace I wrote DVD-R or DVD-RW, I mean +R or +RW.... the Plus formats have error correction and much better data reliability than the Minuw formats. I don't use anything else, ever, so I neglected to think about it. Sigh.

(!) [Pete] That's a great article Brian - enjoyable to read, easy to follow, and made me want to try it out for myself after reading :)
    ___      __           __               ____
   / _ \___ / /____   __ / /__ _    _____ / / /
  / ___/ -_) __/ -_) / // / -_) |/|/ / -_) / /
 /_/   \__/\__/\__/  \___/\__/|__,__/\__/_/_/
(!) [Thomas] Yes, it was food, wasn;t it? Is ther anyew reasin, Pete, why you;'re .sig file is using figket? Only it is ridiculouslt huge.
(!) [Pete] Lol - it is isn't it, ridiculous and huge :) (not to mention that it breaks the convention of the 4 line limit to .sigs)
(!) [Thomas] It's a 5 libe limnit.
(!) [Pete] I've always wanted to implement a witty .sig generator, but don't have the patience to build the data file. You're lucky I didn't use cowsay instead (and I was tempted) :D
(!) [Thomas]
< moo >
       \   ,__,
        \  (oo)____
           (__)    )\
	      ||--|| *


/ So of course, then, the only answer is  |
| to use cowsay in the *body* of the      |
| message, which doesn't violate any .sig |
| etiquette at all, while still being     |
\ annoying and capricious.                /
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
	        ||----w |
                ||     ||

(!) [Sluggo]
< Yeah, annoying, heh heh! >
   \         __------~~-,
    \      ,'            ,
          /               \
	 /                :
        |                  '
        |                  |
        |                  |
         |   _--           |
         _| =-.     .-.   ||
         o|/o/       _.   |
         /  ~          \ |
       (____@)  ___~    |
          |_===~~~.`    |
       _______.--~     |
       \________       |
                \      |
              __/-___-- -__
             /            _\

(?) India currency question (/Flash sucks)

From Sluggo

I was curious about this quote in a Slashdot article: "~$350,000 (Rs. 1.5 crores)". Is a crore a multiple of rupees? Or is this a typo?


(!) [Suramya] One Crore = 10,000,000 It goes something like
1            Ones
10           Ten
100          Hundred
1,000        Thousand
10,000       Ten Thousand
100,000      Lakh
1,000,000    10 Lakhs
10,000,000   Crores
100,000,000  10 Crores
(!) [Kapil] Crore = 100 Lakhs = 100 00 thousand.
For some reason, the Indian system of counting goes in 100's after 1000. (Check it out on wikipedia).

(?) Warning: the India Times URL in the article crashes my Firefox. Must be the Shockwave ads.

(!) [John] I suppose, since the FF 1.0Pre I use handles it - probably because the installed flashblock extension is preventing it.
(!) [Ben] No problem on my end -
ben@Fenrir:~$ mozilla -v
Mozilla 1.7, Copyright (c) 2003-2004 mozilla.org, build 2004061606
(!) [Jimmy]
Dammit, I love Firefox. There's an extension called GreaseMonkey that allows you to install user defined javascripts to modify individual websites, such as this one that not only replaces Flickr's flash interface, but improves on it!

See attached lickr.user.js.txt

(?) Hydrogen power

From Ben

Funny "small world" coincidence: we had a discussion on hydrogen powered cars, etc. here a while ago - and I just got an article submission from a fellow who works at Risø, as the webmaster for their Hydrogen Storage Group.

Published in Issue 114 of Linux Gazette, May 2005