Slothoughts Hypolite

That's the prescription we have been given for swine flu, or as we now like to call it, so as not to offend the sensibilities of our porcine brothers and sisters, H1N1.

Of course what we were really told was to wash our hands endlessly, for a full 20 seconds at a time. If you actually look at a watch for 20 seconds, you'll see that is a rather long time. So long, in fact, that you can't even come close to guessing it. Thus, you need to find something to say or do while you're washing your hands all that time. Something that takes a full 20 seconds to say or do. That something turns out to be two complete Hail Marys.

Now, given that I was born and bred a Protestant, one of the more liberal Protestant denominations actually (née UPC USA; currently UCC), how would I know about Hail Marys?

Well, it seems that when I was in 7th or 8th grade, I used to listen to the radio a lot. I had rather a good one, a Hammarlund HQ100 A. I liked changing stations on both AM and shortwave to see what I could find. I would listen to stations from Buffalo, Nashville, Cleveland, Boston, Toronto, Moscow, Quito. You name it, I probably heard it at one time or another.

One day, I chanced upon a group reciting the rosary. Over and over again they chanted:

Hail Mary, full of grace,
the Lord is with thee.
Blessed art thou among women.
Blessed is the fruit of thy womb,
Jesus Christ, Our Lord.

Holy Mary, mother of God,
Pray for us sinners,
Now and at the hour of our death.
Amen.

I may have this a bit wrong. Seventh or eighth grade was a long time ago. Whatever, when I started washing my hands, and looking at my watch to see if I'd done it for twenty seconds, those words from long ago popped into my head. I discovered that, when said at a measured pace, two full Hail Marys, at least as I had remembered them, took up twenty seconds.

So that's how we combat swine flu H1N1: we spend quality time at the sink saying our Hail Marys. The Pope would be proud of me.

Some of the better off people in my neighborhood, like the guy who appears in a popular public TV production (and the hottie across and slightly down the street from him), can afford to hire "landscapers" to take care of their yards. Thus on a regular basis, Brewster and I, who spend quite a bit of time walking around the neighborhood, checking out the sights and smells, see platoons of little brown people scurrying around these people's lawns.
 
Yes, I know that sounds a bit racist. I also know that I have a son who has worked as a landscaper, so the stereotype isn't completely true. None-the-less, the vast majority of landscapers in my neighborhood are somewhat more swarthy than your average Anglo Saxon person, whose family has lived in the U.S. for half a dozen generations...or more. As nearly as I can tell, they aren't speaking English to each other either. I have no problem with that. That's just how things are in the suburbs north of Boston.
 
So it was quite interesting to me when YSOP assigned my mission trip group to be a landscape crew one day. After a long subway ride out from the center of Washington, D.C., and a bit of a walk, we found ourselves in front of a building managed by Mr. Barry. He has a last name, but I forget it. He likes to be called Barry, but, given that he's clearly the boss of his domain, he is "Mr. Barry" to us lesser folks.
 
Mr. Barry manages a transition home, Milestone Place, housing about 35 people for an organization called Community Family Life Service. He is quite strict with them: no loud noise; no drugs; no alcohol; no overnight visitors; clean up after yourself in common areas; etc. Mr. Barry doesn't take crap from anyone. If you leave your dirty dishes in the sink, figuring you'll do them later, you'll find them in the dumpster.
 
But, Mr. Barry had a problem. He doesn't have much of a budget for upkeep. If you don't keep a place up, the tenants soon get slack and things go to the dogs...so to speak. Mr. Barry, however, is a very smart man. He has found a solution to his problem: volunteers.
 
Yup, Mr. Barry gets volunteers to come out to his place and do maintenance of various sorts. Sometimes it's painting, sometimes repair, sometimes cleaning. For my group, mostly teenage girls, we were to do "yard work". We began with mowing the lawn and trimming the edges. We also did quite a bit of weeding and some general trash pick up in the parking lot. As I was pushing the lawn mower around, I suddenly realized how smart Mr. Barry was. Whereas my neighbors, like the public TV personality (and the hottie across and slightly down the street from him), have to pay good money to get little brown people to keep up their yards, Mr. Barry can get a platoon of little white people to keep up his yard for free.
 
He had something over 1700 volunteers last year, and expects to have even more this coming year to keep things neat and tidy for him. Don't take my word for it. Go visit Mr. Barry. You'll see a picture of the girls and me on his wall, along with pictures of the other several thousand volunteers he's had work for him over the past few years. Just don't forget to bring a rake, broom, or paint brush with you.

       by ngungo and lgpiper

Introduction

When the Ma.gnolia site went down a few weeks ago, most people's data were irretrievably lost. One can learn two lessons from this issue. One is that a robust, off-site back up plan is necessary. The other is that back-ups necessarily need be redundant. Thus, if one back-up becomes corrupt or incapacitated in any way, the data are still available via the second back up. We deal with the first of these two problems: developing a routine that ensures regular back up. Our concern is with moderate sized data bases. We are developing a few small web applications: Monpage.com and bookMarkR.us. They may well grow slowly, but they will never grow at all if we don't protect our users' data from the beginning.

Background

We wish to develop a generalized backup routine. The language of the back up is of little consequence. We have chosen to program our back-up routine in PHP. The core of the back up will be a function that looks something like the following:

 

function dirBK($from, $dest) { 
// This function will recursively copy all files and sub directories 
// from a single directory, $from, on a hosting server 
// to a single directory, $dest, on a remote storage disk. 
 // rsync seems the best candidate for the file-copying protocol. 
}

If there are a lot data in a huge number of directories--e.g. one directory per monpage account, or one directory per site that each account monitors--, using rsync is not without its own problems. It can tie up the i/o ports on the server, then websites--anything else for that matter--on the hosting server would become temporarily unavailable. All activity would freeze whilst waiting for rsync to finish. Depending on the amount of data, this shut-down could take up to several hours. One way to solve the problem is to run the back up in piecemeal fashion, backing up one small directory at a time, with a pause in between each back-up segment. Thus, the function dirBK() must incorporate a loop which spreads the many small back-ups over an extended period of time--e.g. hours, days....

 Another problem with rsync arises where, under a non-ideal scenario such as broadband disruption, rsync hangs and does nothing. We must, therefore, find a solution for this problem too.

 Given the above considerations, we now have three major tasks, each of which may include some subtasks:

1. Developing the dirBK($from, $dest) routine.
2. Developing a general driver to run this routine in a loop.
3. Scheduling the driver (e.g. running the back-up routine regularly using cron.)

This article only deals with the first two tasks because the scheduling task is server specific and also because, once the general routine functions properly, automatic scheduling is relatively simple to implement.

Back-up Routine Development

Developing dirBK is rsync specific and secondary to the topic here, so we will treat it last. Just be aware that it can be disrupted and hold up the whole grand scheme based on our driver routine. We will first describe the driver in simple terms, than add detail as we go along.

 We begin with the variable, $dirs, which is an array of all directories that need to be backed up. The driver routine then becomes:

 

foreach($dirs as $dir) { 
$from = 'path/to/dir/'.$dir; 
$dest = 'path/to/destination/'.$dir; 
dirBK($from, $dest); 
sleep(5); 
}

This seems to be a good approach, backing up each directory, one at a time, and pausing few seconds in between. The pausing time could be a tuning parameter, depending on the efficiency of the i/o. What might still cause problems, however, is the situation in which dirBK() function halts in the middle of the file transfer process. One solution might be if we could somehow run each individual directory back up as a independent process. Thus, if one transfer breaks down, others will still be able to continue normally. To do this, we need to implement process spawning, using the pcntl_fork() function. Our modified routine now becomes:

 

foreach($dirs as $dir) { 
$from = 'path/to/dir/'.$dir; 
$dest = 'path/to/destination/'.$dir; 
$pid = pcntl_fork();    // spawn child process 
if (!$pid) {          // a child process returns zero 
dirBK($from, $dest); 
exit; 
} 
sleep(5); 
}

We further need to check 2 things. First, if we will want to exclude some directories. For example, in a normal directory scanning we should exclude the "dot" and "double dot" directories. Secondly, we must be sure that what we are backing up is, indeed, a directory. So the next version of the routine is:

 

foreach($dirs as $dir) { 
if ($dir == '.') continue;       //exclude '.' and '..' directories 
if ($dir == '..') continue; 
$from = 'path/to/dir/'.$dir; 
if (is_dir($from)) {            //make sure array element is a directory 
$dest = 'path/to/destination/'.$dir; 
$pid = pcntl_fork();          // spawn child process 
if (!$pid) {                // a child process returns zero 
dirBK($from, $dest); 
exit; 
} 
} 
sleep(5); 
}

At the end, the driver quits and all independent backup processes, running as the driver's children, also quit, including those that hung up for whatever reason. The command, sleep(5), gives some breathing room between each backup increment. Sometime, however, five seconds will not be long enough to ensure that the last cycle is complete. Thus, we give it a a little extra time, perhaps an extra 30 seconds.

 Above, we specified an array of the directories to back up, $dirs. There are a number of ways to populate this array. If all of the directories are in the rootBK directory, we can use the scandir() function to populate the array. Our basic routine is now as follows:

 

<?php 
$dirs = scandir('path/to/rootBK/');   //populate the array, $dirs 
foreach($dirs as $dir) { 
if ($dir == '.') continue;    //exclude '.' and '..' directories 
if ($dir == '..') continue; 
$from = 'path/to/dir/'.$dir; 
if (is_dir($from)) {        //make sure array element is a directory 
$dest = 'path/to/destination/'.$dir; 
$pid = pcntl_fork();       // spawn child process 
if (!$pid) {              // a child process returns zero 
dirBK($from, $dest); 
exit; 
} 
} 
sleep(5); 
} 
sleep(30); 
?> 

It now remains to develop the dirBK routine. The whole routine can just a system call to the rsync command:

 rsync -r /path/to/existingsite/ username@newsite.com:/path/to/newsite/ 

It may be necessary to extend this with a few more parameters to specify a password file:

rsync -azq --delete -e "ssh -i /users/home/myaccount/.ssh/ss" /path/to/backups/ myaccount@myaccount.strongspace.com:/home/myaccount/backups

We won't go into the details of the rsync command, but you can find them on line at the following urls:

• http://forum.textdrive.com/viewtopic.php?id=10126
• http://wiki.joyent.com/shared:kb:automated-backups (subtitle rsync to Strongspace)

We will use the second method above, i.e. with the password file specified. To break up the command into more comprehensible components, we have,

$rsync = '/path/to/rsync -azq --delete -e "ssh -i /.ssh/ss"'; 
   $from = '/path/to/backups/'; 
   $remote = 'myaccount@myaccount.strongspace.com:/home/myaccount/backups';

Then, as mentioned above, we embed these variables into a php system call, remembering that we need a space between the three strings, $rsync, $from, and $remote:

system($rsync. ' ' .$from. ' ' .$remote); 

Placing this call into the main scheme produces the final version:

<?php 
$rsync = '/path/to/rsync -azq --delete -e "ssh -i /.ssh/ss"'; 
$remote = 'myaccount@myaccount.strongspace.com:/home/myaccount/backups'; 
 $dirs = scandir('path/to/rootBK/'); 
foreach($dirs as $dir) { 
if ($dir == '.') continue; 
if ($dir == '..') continue; 
$from = 'path/to/dir/'.$dir; 
if (is_dir($from)) { 
$dest = $remote.$dir; 
$pid = pcntl_fork(); 
if (!$pid) { 
system($rsync. ' ' .$from. ' ' .$dest); 
// echo $dir; 
exit; 
} 
} 
sleep(5); 
} 
sleep(30); 
?>

The echo $dir command was added for debugging purposes. To begin, one might wish to comment out the system call and un-comment the echo line. Then when the routine appears to be processing the expected directories properly, reverse the comments so that the full back-up is working.

 If the name the file containing our routine backUp.php, we can run it manually from the command line, making sure to be in the proper working directory:

# php5 backUp.php

Sometimes it is necessary to specify the full path to the php command, and also the php.ini file. In that case the command might look as follows:

/usr/local/bin/php -c /users/home/userName/etc/php5/ /users/home/userName/backUp.php

The specifics, naturally, depend on one's particular server configuration.

 Once the routine works as intended from the command line, then one should write a cron job to automate the process. That is a subject for another time.

I said something in my class a couple of weeks ago about visiting a Buddhist temple in Wakefield along with a comment on the art. That was a mistake, of sorts, in that my teacher, Catherine, then wanted me to post a link or two (which means work for me). Anyway, my friend, Roy, has a flickr page where he has posted photos of some of the art work. His photo stream has a picture of the dharma hall with the hanging lotus lanterns.
 
Every year, they tear down and remake all these lanterns. My attempts at paper art never got past folding origami cranes, so I found the description of the lantern making process quite fascinating. Roy blogged the process in steps: one, two, three, four.

Several years ago, it occurred to me to create a Psalm-o-matic for Lent. I have no idea why this occurred to me, it just popped into my head one day and I couldn't forget it. In theory, we are supposed to be a bit more focused on spiritual things, like scripture and prayer, during Lent. Just because we're Congregationalists—UCCs actually—why shouldn't we attempt to bow to tradition, at least a bit. So to make the scripture part a little more palatable—or so I thought at the time—how 'bout if we each read a Psalm each day? Rather than expect people actually to hunt up a Bible, and then figure out where the Psalms lay in that book, why not just e-mail out a Psalm? If we're to read Psalms, why focus on the three or four we already know. How 'bout we read a randomly selected Psalm? That way, in theory at least, we would get through Lent with a representative selection of Psalms, the familiar and the not-to familiar.

 Thus was born the idea of the Psalm o' matic. Now to figure out how to implement it. Of course, I thought it would be cool to write a computer program to do the work for me. I had a job at the time, and was a bit rusty on my programming, so I ended up with the coward's way. I used the random-number generator function in my spread sheet program to make up a list of numbers between 1 and 150. Then I copied the list over to a schedule, printed it out, and manually e-mailed out the Psalm scheduled for each day. Not all that o' matic, huh?

 I did that for a couple of years, but this year, since I was unemployed, I figured that I should brush off my very rusty shell scripting and try to make a real, bone fide psalm-o-matic. The shell script would, in theory at least, randomly pick out a number from 1 to 150, select the Psalm that related to that number, and do the e-mailing for me. All while I slept.

 The first thing I needed to do was come up with a method for generating random numbers from 1 to 150. Of course that's trivial if you are programming in C or C++, or using a spread sheet, but I was trying to do this in Unix shell scripting, the only way I knew how to send out e-mails automagically. It turned out that what I knew about sending out e-mails—learned in Intro to Unix back in the summer of 2000—was no longer valid, at least not on my system. But I didn't know that when I began this project.

 Anyway, it turned out that generating random numbers from the unix shell was relatively easy. My server at Joyent runs bash-shell scripts, and the bash shell has a build-in shell variable called $RANDOM. $RANDOM returns a random number from 0 to 32767 whenever it is called. To pare this selection down to 1 to 150, I merely had to multiply the $RANDOM variable by 150, divide by 32767, and then add 1 to take care of the fact that Psalms begin with 1 not 0 (zero).
psalmNo=$((1+150*$RANDOM/32767))
and the variable, $psalmNo would indeed be a number between 1 and 150.

 So far, so good. Now I just needed to select the appropriate Psalm and send it off. Selecting the appropriate Psalm was trivial, albeit tedious. I made up 150 text files with names like psalm_xxx.txt, where xxx was a number from 1 to 150. Then I just needed to combine three elements, the root of the text-file name, the Psalm number and the extension:
body="$filePath$fileRoot$psalmNo$fileExtension"
Oh, yeah, I forgot to mention, the $filePath thing tells the computer where to find the directory that has all the Psalms in it.

 So now we get to what I assumed would be the easy part, e-mailing the Psalm. In olden days, we did something like the following:
mailx -s "e-mail subject" someone@somedomain.com <$body
That is, the file in $body is inserted in an e-mail message to someone with the subject given. Well, it didn't work. After thrashing around for quite some time, I asked for help and was told that mailx no longer worked because of security issues. I had to use sendmail -t instead.

 I'd never heard about sendmail -t before. I guess it didn't work in the olden days. Or perhaps it just didn't work at UMass/Lowell, who tend to be a bit behind the times (when I took Java Script, I was told to be wary about things that caused problems with Netscape 2.0. WTF? we were in a post-Netscape era by the time I took that class.)

 Anyway, I had to learn about sendmail. That wasn't overly difficult. I discovered that the -t option meant that the system used the header information embedded in the file. So eventually, I figured out that meant I had to make up the files I e-mailed out with stuff like "To: someone@somedomain.com" at the top of the file, and the text file containing the Psalm at the end. Well, crap, I didn't want to go through 150 files and put in all the header stuff first. What if I wanted to do something else with the files later on?

 Fortunately, this wasn't a problem. I suddenly remembered about "appending" things to a file. To append things, you just did a double carat thingie, i.e.
cat $body<<$filePath$mailMsg
and you get the stuff in $body, i.e. the Psalm file, appended to $mailMsg, the thing you created with all the header info. So I build it up a bit at a time, starting with the "To: someone@somedomain.com" stuff, adding in the From, Reply-to, and Subject stuff and finally ending with the Psalm file.

 So after all this ruminating, we ended up with

# !/usr/local/bin/bash 
# psalm o' matic script 
# 
#  This works as follows: me@myServer$ /usr/local/bin/bash psalm-0-matic 
#  If you don't precede with the "bash" it won't work. 
#  The fully qualified paths through out are necessary to make this work as a cron script 
# 
#    This uses my random number generator script 
# 
#    Why in the hell am I using vi? 
#    I'm not anymore -- using TextPad with sftpDrive (now ExpanDrive) 
 filePath="/users/home/myUserName/domains/myDomain.org/web/public/Psalms/" 
fileRoot="Psalm_" 
fileExtension=".txt" 
 # ***seed Random with a "random" number -- then select Psalm o' day 
 RANDOM=$RANDOM 
 psalmNo=$((1+150*$RANDOM/32767))  #all 150 -- Oh, yeah! 
 #  *****************prepare the message ****************** 
 mailMsg="MailMsg" 
 today=`/usr/xpg4/bin/date '+%A, %B %d, %Y'` 
 #  ***************** prepare header header ****************** 
 dateField="Date: "$today 
#toField="To: me " 
#later on it would be fccr-ace. 
addressFile=pomList 
toField="To: "`cat "$filePath$addressFile"`  #pomList is comma delimited: name , ... 
fromField="From: me " 
replyToField="Reply-to: me@myOtherDomain.net"  #later on it would be fccr-ace? 
subjectField="Subject: Psalm for "$today"--Psalm "$psalmNo 
 #  ***************** assemble message ****************** 
body="$filePath$fileRoot$psalmNo$fileExtension" 
 echo $dateField<$filePath$mailMsg 
echo $toField<<$filePath$mailMsg 
echo $fromField<<$filePath$mailMsg 
echo $replyToField<<$filePath$mailMsg 
 echo $subjectField<<$filePath$mailMsg 
echo " " <<$filePath$mailMsg 
echo " " <<$filePath$mailMsg 
cat $body<<$filePath$mailMsg 
 #  *****************send the message ****************** 
 /usr/local/bin/sendmail -t <$filePath$mailMsg 

 Ok, this worked. Getting it to send out automagically turned out to be more trouble than I realized, but I eventually got so I was sending myself Psalms on an hourly basis. Just in time for Lent.

 Addendum:
The scheme worked like a charm and we lucked out in that Ps. 119 never came up. Not sure anyone would have been able to slog through that one in one sitting.

 Addendum 2:
Formatting this was a real PITA. I'd have been better off just doing raw xhtml on my static site. Then again, I suppose that I needed to (re?)learn that'code' is not a block-level element.

... That's what we sang at the Maundy Thursday service tonight. I have always loved this piece. I don't think we were so great, but then again, we probably weren't that bad. We recorded a passable version of it a few years back, which is reproduced here.
 
Because we're Congregationalists, we can sing in Latin. That seems weird, since our denominational roots are decidedly anti-Catholic (think Puritans). But for quite some time after Vatican II in the mid 60s, Catholics weren't allowed to do Latin. So you had to visit Congregationalists and Presbyterians and the like to hear things in Latin. I think that may have changed in the past few years.
 
The lyrics we sang:
Ave, ave, verum corpus
natum de Maria Virgine,
Vere passum immolatum
in Cruce pro homine,
Cujus latus perforatum
unda fluxit et sanguine,
Esto nobis praegustatum
in mortis examine.
 
Translation:
Hail,true body
born of the Virgin Mary,
Who truly suffered, sacrificed
on the Cross for man,
Whose pierced side overflowed
with water and blood,
Be for us a foretaste
In the test of death.

(download)

Once I was baking cookies with my older sister. We were listening to the radio while we baked. The songs were typical teen-ager kind of stuff for those days. Then, amazingly, the DJ played a song for which he wanted his listeners to provide feed back. The kitchen was suddenly filled with the sounds of I Put a Spell on You, by Screamin' Jay Hawkins. I guess the other listeners weren't so impressed as I was. I thought the song was amazing, but I never heard it again until I was an adult with kids of my own. One of my kids helped me find this copy of the song.

(download)

...Paul Robeson, or the bass guy in the Harmonizing Four, Jimmy Jones. Wouldn't it be cool if one or both of them came to choir? Or course ol' Paul has been gone for more than 30 years, and I expect the Jimmy is long gone as well. There are people who sing in the bass section of choirs, and then there are Paul Robeson and Jimmy Jones. There are people who can swim and then there is Michael Phelps (who went to my high school). Where did I go wrong?

(download)

(download)

(download)

(download)

(download)

(download)

(download)