HOWTOS and White Papers

Got questions/comments/corrections? Email me (mbates _a_t_ whoopis _d-o-t_ com).

Entries with the modification date in red have been posted or updated sometime in the past week.

Email with sendmail and IMAP: [html] Last modified: April 19 2005 14:15:17
See the Unix man pages for sendmail here.
See the Unix man pages for imap here.

Setting up sendmail with auth and ssl: [html] Last modified: April 19 2005 14:16:51
See the Unix man pages for sendmail here.

Mailman Quickstart for RedHat Linux 9: [html] Last modified: April 19 2005 14:16:44

Spamassassin Quickstart: [html] Last modified: April 19 2005 14:15:48
See the Unix man pages for spamassassin here.

Spamassassin 3.0 RedHat Linux Installation Guide: [html] Last modified: April 19 2005 14:16:54
See the Unix man pages for spamassassin here.

Web hosting with Apache VirtualHosts: [html] Last modified: April 19 2005 14:16:58
See the Unix man pages for httpd here.

Using Apache realms to password-protect your website: [html] Last modified: January 09 2006 12:04:02
See the Unix man pages for httpd here.

Using Apache's Rewrite Engine to rewrite URLs, redirect to https:, and play nice with realm authentication: [html] Last modified: September 17 2007 12:15:41
See the Unix man pages for httpd here.

Webserver Bandwidth Limiting in Apache: [html] Last modified: September 19 2005 12:34:01

Linux NFS Crash Course: [html] Last modified: June 11 2007 12:57:56

Working with Multiple LVM Hard Disks under Fedora Core Linux: [html] Last modified: October 26 2007 18:02:13

Using dummynet to Simulate a Low-bandwidth Network Connection over Two Ethernet Interfaces: [html] Last modified: February 05 2007 14:39:06

iptables firewall quickie: [html] Last modified: July 27 2006 16:44:03

How to Restore GRUB and/or the MBR (Master Boot Record) under Fedora linux: [html] Last modified: July 29 2005 11:56:08

How to install different architecture binaries on Linux: [html] Last modified: April 19 2005 14:15:28

Linux system management with Ximian redcarpet: [html] Last modified: April 19 2005 14:16:50
See the Unix man pages for rcd here.

DNS with BIND/named: [html] Last modified: October 10 2007 12:08:38
See the Unix man pages for named here.

Secondary (slave) DNS with BIND/named under Fedora Core 6: [html] Last modified: October 10 2007 12:09:15
See the Unix man pages for named here.

Setting up automatic alerting in your Unix environment: [html] Last modified: April 19 2005 14:16:56

Setting up a RedHat Linux/Windows 98 Dual-boot System: [html] Last modified: April 19 2005 14:15:16

Upgrading from Redhat 7.3 to RedHat 9 remotely: [html] Last modified: April 19 2005 14:16:50

Linux tape backup basics howto: [html] Last modified: April 19 2005 14:16:57
See the Unix man pages for mt here.

Adding a second hard disk under Windows: [html] Last modified: December 13 2005 13:09:02

How to Fix PNG Support in MediaWiki: [html] Last modified: June 11 2007 12:44:17
See the Unix man pages for php here.

PHP 4 and Sablotron XSLT: [html] Last modified: April 19 2005 14:16:47
See the Unix man pages for php here.
See the Unix man pages for rpm here.

PHP 5 and MySQL 4 on Fedora Core 3, from RPMs: [html] Last modified: April 17 2006 10:13:23
See the Unix man pages for php here.
See the Unix man pages for mysql here.
See the Unix man pages for rpm here.

MySQL Crash Course: [html] Last modified: May 16 2008 10:00:21
See the Unix man pages for mysql here.

How to fix "Client does not support authentication protocol requested by server; consider upgrading MySQL client" problem: [html] Last modified: September 28 2006 15:23:07
See the Unix man pages for mysql here.

bBlog Quickstart: [html] Last modified: April 19 2005 14:15:11
See the Unix man pages for mysql here.

MySQL ssh tunnel quickstart: [html] Last modified: April 19 2005 14:16:46
See the Unix man pages for ssh here.

Setting up php and mysql (and phpbb) on Mac OS X 10.2: [html] Last modified: April 19 2005 14:16:47

Sample PHP code: [html]

How to Use Audio Hijack to Record a RealPlayer Stream: [html] Last modified: June 11 2007 12:44:20

Making Resource-Fork-Aware Backups with rsync on Mac OS X: [html] Last modified: April 12 2007 17:01:54

Using a Lucent Wireless Card under Mac OS X 10.3 Panther: [html] Last modified: July 27 2006 17:13:02

Exporting multi-platform-compatible QuickTime movies from Apple iMovie: [html] Last modified: May 11 2005 14:58:02

A Dartmouth Mac FAQ (needs updating): [html] Last modified: September 16 2005 12:51:13

Importing self-signed certificates in Mail on OS X: [html] Last modified: April 19 2005 14:16:44
See the Unix man pages for ssl here.
Click here to get current sendmail and imap ssl certificate blocks for whoopis.com and associated domains. Last modified: November 08 2005 15:07:58

Macintosh Security Basics presentation: [html] Last modified: April 19 2005 14:16:22

Setting up php and mysql (and phpbb) on Mac OS X 10.2: [html] Last modified: April 19 2005 14:16:47

MacSSH SSH2 Key Exchange Howto: [html] Last modified: April 19 2005 14:15:58

How to Reset the Configuration of a used Apple LaserWriter 16/600 printer: [html] Last modified: April 19 2005 14:15:53

How to set up Ray Arachelian's LisaEM Lisa Emulator under Mac OS X: [html] Last modified: April 12 2007 15:11:43

How to Make Old Floppy Diskettes in the Internet Age: [html] Last modified: April 25 2007 16:34:44

Apple repair manuals: [html] Last modified: July 27 2006 16:39:18

Using CVS over SSH: [html] Last modified: April 19 2005 14:15:16
See the Unix man pages for cvs here.

Secure Chat using privoxy and ssh: [html] Last modified: April 19 2005 14:15:08
See the Unix man pages for privoxy here.

MySQL ssh tunnel quickstart: [html] Last modified: April 19 2005 14:16:46
See the Unix man pages for ssh here.

MacSSH SSH2 Key Exchange Howto: [html] Last modified: April 19 2005 14:15:58

iptables firewall quickie: [html] Last modified: July 27 2006 16:44:03

phpBB viewtopic.php Vulnerability Hack and Forensic Followup: [html] Last modified: December 13 2005 13:57:15

An incident handling analysis: [pdf] Last modified: April 19 2005 14:15:10

Macintosh Security Basics presentation: [html] Last modified: April 19 2005 14:16:22

Introduction to HTML: [html] Last modified: April 19 2005 14:15:45
See the Unix man pages for http here.

Intro to HTML in depth: [html] Last modified: April 19 2005 14:15:47

Using Javascript to Preserve Form Contents During Browser Refresh: [html] Last modified: April 19 2005 14:15:47

photoPopup Quickstart: [html] Last modified: November 21 2006 14:33:42

Changing the LaserJet Ready Message: [html] Last modified: July 27 2006 16:58:06

Rounded-corner rectangles in Photoshop: [html] Last modified: December 15 2005 12:50:19

Creating Standalone Tcl/Tk Applications: [html] Last modified: May 02 2005 17:02:06

How to load a custom map in Unreal Tournament 2004 for Mac OS X: [html] Last modified: April 19 2005 14:15:58

BitTorrent Quickstart: [html] Last modified: April 19 2005 14:15:12
See the Unix man pages for bittorrent here.

Running OpenMCU (OpenH.323 conference server) under RH9/Fedora: [html] Last modified: April 19 2005 14:16:47
See the Unix man pages for openmcu here.

sshlitzMail!: Creating and Using a Secure, Encrypted Tunnel to Access the Dartmouth BlitzMail System: [html] Last modified: April 19 2005 14:16:54

BlitzMail for Linux: [html] Last modified: April 19 2005 14:16:52

Get Euro-style oval stickers for Geeks!

NEW -- the "magic/more magic" light switch cover!
Click here for the story.

Home phone wiring basics: [html] Last modified: October 03 2006 10:03:02

Miscellaneous perl, bash, and PHP scripts and snippets.
See the Unix man pages for perl here.

makelinks.pl:

#!/usr/bin/perl -w
# makelinks.pl by Marion Bates 11-15-02. 
#
# This is a script to take a list of things (each on separate line)
# and write a new file with each of those things printed as a link
# to themselves. In other words, if your starting list file contains
#
# 	foo
# 	bar
#	foobar
#	barfoo
#
# and you run this script on that file, it will generate a new file
# called "linkedlist.txt" which says:
#
#	<a href="foo">foo</a>
#	<a href="bar">bar</a>
#	<a href="foobar">foobar</a>
#	<a href="barfoo">barfoo</a>
#
# USAGE: Place this script in the same directory as your file, and
# change $TheFile to be the name of that file. Then run this script
# by typing ./makelinks.pl
#

$TheFile = "yourfile.txt";		# Change this part
open (INFILE, $TheFile) or die "The file $TheFile could not be opened.\n";
open (OUT, '>linkedlist.txt') or die "Oops, I can't make a new file...check permissions?\n";

while (<INFILE>) {
	$TheLine = $_;		# Tell Perl to assign the next line to our variable. $_ means "line"
	chomp($TheLine);	# Remove line breaks, we'll put them back in a sensible way
	$TheLine = "\n<a href=\"$TheLine\">$TheLine</a>"; 		# Note the escaped quotes
	print (OUT $TheLine);		# Write it out
}	# End while-loop 
	
# End of program

fix-linefeeds:

#!/bin/bash
# Fix Macintosh linefeeds so the files display properly in a Unix text editor
# Usage: ./fix-linefeeds.sh fileyouwanttofix.txt

exec `perl -pi -e 's/\r\n?/\n/g' $1`

#end

iFind:
See the Unix man pages for bash here.

#!/bin/bash

# Stupid little script for quickly finding songs under the strange hierarchy of the 
# iPod "Music" folder (un-hidden via "iPod MP3 Enabler")

# by Marion Bates <mbates@whoopis.com>
# Last updated March 31, 2004

IPODNAME="$1"
FILENAME="$2"

# Count args
if [ $# -ne 2 ]
then
	echo -e "\nWrong number of arguments, exiting..."
	echo -e "Usage: $0 name-of-ipod songname-fragment (return)"
	echo -e "If your iPod name has spaces or punctuation in it, enclose its name in quotes, or rename it, please." 
	exit 0
fi

# Test if the iPod they named is actually there
PODTHERE=`ls /Volumes/ | grep $IPODNAME`
if [ "$PODTHERE" = "" ]
then
	echo -e "\niPod named $IPODNAME not found under /Volumes/, exiting..."
	echo -e "If your iPod name has spaces or punctuation in it, enclose its name in quotes, or rename it, please."
	exit 0
fi

# Test passed? Move on
PODPATH="/Volumes/$IPODNAME/Music"
echo -e "\nSong names containing $FILENAME are present in the following directories under\n$PODPATH:\n"

# Case-insensitive find 
# (NOTE: It needs that trailing slash for some reason, "find" is buggy maybe)
find "$PODPATH/" -iname *$FILENAME*

# Let user decide whether or not to pop open a folder in the Finder
echo -e "\nFolder to open? (E.g. \"F12\") or enter \"n\" for none."
read FOLDER

# Not bothering to test if the folder name is valid; the open command 
# already exits cleanly if the user enters something bogus.
if [ "$FOLDER" = "n" ] || [ "$FOLDER" = "N" ]
then
	exit 0
else
	open "$PODPATH/$FOLDER"
fi

# Alternate 'find' lines:
# show _just_ the folder name (char positions 3, 4, and 5) 
# find "$PODPATH/" -iname *$FILENAME* | cut -c 3,4,5 

# auto-open (bad idea without quantity-checking results; could open all the folders multiple times)
# find "$PODPATH/" -iname *$FILENAME* | cut -c 3,4,5 | xargs open

# end script

Batch renamer:
See the Unix man pages for bash here.

#!/bin/bash
# Stick this in a dir full of files that end in
# .php and it will rename them all to .phps 

for file in *.php ; do mv $file `echo $file | sed 's/\(.*\.\)php/\1phps/'` ; done

make IP addresses into links to themselves with sed: In a file containing dot-quad notation IP addresses, this will turn each of them into an HTTP link to themselves. In other words, if the file contains

blah blah 10.1.2.3 more blah blah 10.2.5.4

then this script will make that be

blah blah 10.1.2.3 more blah blah 10.2.5.4

cat ip-addresses.html | sed 's/[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*/<a href="http:\/\/&">&<\/a>/g' > ip-addresses-links.html

burn-cd bash script:

#!/bin/bash
# Usage: ./burn-cd </path-to/image.iso>

if [ $1 eq ''] ; then

        echo -e "\nNo input file specified.\nUsage: ./burn-cd /path-to/image.iso\n"
        exit

else
        sudo nice --18 cdrecord -v speed=4 dev=0,0,0 $1
fi

Remote SSH port forwarding:

your_home_mac% ssh -R 2222:localhost:22 -g mbates@whoopis.com

.... will make whoopis.com listen on port 2222 and forward it over an encrypted tunnel to port 22 on your home Mac.  This is useful if the latter is totally firewalled or 192.168'ed.

If you get rid of "-g", whoopis will only listen on the loopback interface, which is a lot more secure.  (Then you can ssh to whoopis.com and type "ssh -p 2222 localhost".) 

If you add "-f -N" it'll detatch and do nothing but tunnel.

Redirection: In this order: 1) redirect stdout with >/some/file , 2) make stderr go to the same place as stdout with 2>&1, and then run it in the background with &

e.g. /path/to/python /path/to/script > /tmp/logfile 2>&1 &

Basic use of dd to clone a hard disk:

dd if=/dev/hda of=/dev/hdb bs=512

Single-user-mode root password reset:

(re)boot, interrupt lilo/grub and add arguments to the kernel line:

init=/bin/bash

It should partly boot and dump you to a root shell. Do:

mount -o remount,rw /
passwd (follow prompts to reset)
sync
mount -o remount,ro /
sync

And then reboot normally.

Find/change in VI:

:%s/search/replace/g

the g at the end replaces all instances

sed note:

echo 'c:\docsandsettings\foo\bar' | sed -e 's/^.*\\//'

result:

bar

Reading contents of a Mac "text clipping" from the Command Line:

If the clipping is called "snippet", do:

cat /path/to/snippet.textClipping/..namedfork/rsrc

Recursive find/change aka search and replace:

find ./ -type f | xargs perl -pi -w -e 's/SEARCH/REPLACE/g;'

    -e = execute the following line of code.
    -i = edit in-place
    -w = write warnings
    -p = loop (?)

Escape any special characters such as forward slash and . E.g. to remove all instances of absolute linking in a directory of webfiles, recursively (descend subdirectories):

find ./ -type f | xargs perl -pi -w -e 's/http:\/\/www\.somedomain\.com\///g;'

I.e. replace "http://www.somedomain.com" with nothing, and we had to escape the slashes and dots.

More variations (based on filename suffix):

find . -regex .*\.htm | xargs perl -pi -w -e 's/http:\/\/www\.ists\.dartmouth\.edu\///g;'
find . -regex .*\.html | xargs perl -pi -w -e 's/http:\/\/www\.ists\.dartmouth\.edu\///g;'
find . -regex .*\.php | xargs perl -pi -w -e 's/http:\/\/www\.ists\.dartmouth\.edu\///g;'

Fixing backslashes in Windows paths:

#!/bin/bash

# Takes 
# 	Foobar\bar\foo\something
# and turns it into just:
# 	something

for file in * ; do 
	# echo "$file" | sed 's/^.*\\//'
	mv "$file" `echo "$file" | sed 's/^.*\\\//'`  
done


#^ = start at beginning of line

#. = any char

#* = zero or more times -- "greedy", so will take as many as it can

# have to double-escape the backslash in the actual execution line, because...I'm in backtics?

Using find to move/copy/etc. a whole bunch of files:

For example, to copy all files in the current directory that end in .php, to a destination directory called /home/foo, do:

find . -iname *.php -exec cp '{}' /home/foo/ ';'

Apache web log to RSS:

#!/usr/bin/perl -w
#
# Reads an apache weblog-format log file and provides an RSS entry for each valid line.
# based on "log2rss.pl" by codepoet (codepoet@codepoetry.net)

use strict;

my $filename = shift @ARGV;
my $cap = shift @ARGV || 0;

if (!$filename) {
print <<END
Usage: log2rss.pl [filename] [max recent lines]
END
;
}

if (-f $filename) {
	my ($file, $line, @lines);
	open(LOG, $filename) || die("Could not open $filename: $!");

	print <<END
<?xml version="1.0"?>
<rss version="2.0">
	<channel>
		<title>$filename -- last $cap lines</title>
		<link>file://$filename</link>
END
;
	my @file = <LOG>;
	for (my $ln = (@file - $cap); @file > $ln; $ln++) {
		$line = $file[$ln];
		$line =~ s/[^[:print:]]//g;
		my ($ip, $datetime, $request, $code, $something, $referrer, $useragent) = $line =~ /^(.*?)\s-\s-\s+(.*?)\s"+(.*?)"\s+(.*?)\s+(.*?)\s"+(.*?)"\s"+(.*?)"+(.*)$/;
		if ($ip) {
			print <<END
		<item>
			<title><![CDATA[$request]]></title>
			<link>file://$filename</link>
			<description><![CDATA[$line]]></description>
		</item>
END
;
		}
	}

	print <<END
	</channel>
</rss>
END
;	
	close (LOG);
} else {
	print STDERR "$filename does not exist.";
}

Continuous feed of logfile to netcat listener:

On the host whose logfiles you wish to see:

while : ; do tail --lines=0 -f /var/log/httpd/www-access_log | nc -l 5555 ; echo -n . ; done

nc logfilehost 5555

ucc-DM: an updated SysV init script for Unreal Tournament 2003 Dedicated Server (Linux): [text] Last modified: April 19 2005 14:16:57

Links

All kinds of good Linux info, software, etc. [link]

Process Query Systems, Inc. Security Monitoring Software [link]