My New iPod

My wife and I don’t really celebrate Valentine’s Day. I’m not sure why. I think it started because, for the first couple of years we dated, we were a few hundred miles apart, so we couldn’t. Now it’s just tradition. Anyway, to make up for it, we go all out on our anniversaries. We had several anniversaries before we were married, but this is our second wedding anniversary, and my wife got me at 160GB black iPod Classic :-D

This is the exact model I wanted. I did not want the iPod Touch. I think the iPod Touch is an insult to my intelligence to tell you the truth. I don’t know why Apple thinks I’m stupid enough to not realize that the iPod Touch is going to magically grow a phone over the next year or two. I don’t know why they think I should spend more money to get 1/10 the storage capacity of the iPod Classic.

The real kicker is that the iPod Touch, which has 1/10 the storage capacity of the iPod Classic, has a direct line to the iTunes Music Store, but the iPod Classic – the social icon around which people build their identities these days, and which has 160GB capacity, doesn’t. Seems like Apple could just put a wi-fi radio in the iPod classic, give it a direct connection to the music store as well, and watch the kids stand around on the playground listening to each other’s tunes and downloading a copy of their own for $.99. Hell, Apple could probably eat 75% of the cost of the data connection, further subsidize it through deals with the ISP, perhaps a small fee built into the initial cost of the iPod, or a $.05 surcharge on songs downloaded directly to the iPod or something, charge the customer *once*, for the iPod instead of a monthly data transmission fee or whatever, and still make money.

For $50 more than the 16GB Touch, you can get the iPhone. It only has 8GB capacity, but replaces two gadgets you may carry around all the time. Your phone, and your point-n-shoot digital camera. I’m not a fan of Apple’s stance toward users who would like to run software not made by Apple on the hardware they allegedly own, but if we’re just talking value and comparing the devices in their line, I have to believe the iPhone is a slightly better value than the iPod Touch.

Anyway, this was really meant to be a glowing review of my iPod Classic. It works wonderfully. I’ve already loaded a few months worth of home brewing podcasts, all the music that I care about (for now – more coming), a bunch of photos of my wife and daughter and stuff, and I’m all set to go. My wife must truly love me. At some point I hope to figure out why. :-)

Sometimes it’s just about attitude

Today I spent the day completely breaking down and replacing the entire above-ground portion of my in-ground pool’s plumbing. I had some problems crop up with my multi-port valve, and I had to take a few things apart to troubleshoot and remove the pump as the source of a pressure problem (namely, there wasn’t any).

Without going into too much gory detail about the troubleshooting, suffice to say that, over the course of the past few weeks, I’ve spent a great deal of time in the pump house. More than I ever had to before. While I was in there I had some time to closely inspect the plumbing. While taking a break and sipping a home brewed scotch ale, I just sorta stared at each component in the line and thought about the logic employed in the overall project. By the end of my 5-minute break, I was not happy.

This plumbing was put in place by a contractor that was hired by the previous owner of this house. Thinking about that got me really steamed for two reasons;

1. I know the previous owner of this house, and he’s a great guy.

2. The contractor completely fleeced the previous owner :-/

The pipe coming out of the ground is 1 1/4″. It went downhill from there. Connected to the black pipe coming from the ground a 1 1/4 barb/thread adapter, and held in (in part) by a worm clamp. Actually – two worm clamps. And guess what screwed into there? A 1 1/4″ thread/barb adapter! Shoved onto *that* with two worm clamps was another length of the black pipe, which went to a very sharp, and *iron* threaded elbow joint which had a 1 1/4 thread/barb bit attached to connect the pipe. Yes, more worm clamps. From there there was an enormous, old, and broken brass or iron valve, and then we see our first bit of actual PVC – a union going into the pump.

Everything in that last paragraph has been completely replaced with PVC.

From the pump to the multi-port valve, there’s a PVC elbow that’s still in tact, but it had a simple barbed adapter, to which was attached 1 1/2″ vinyl tubing…. with worm clamps. The vinyl tubing connected in the same exact manner on the multi-port valve side.

Everything above was replaced with PVC, and the connections to the valve and pump are now either threaded or glued – but not barb fittings, and there are no worm clamps.

The return line (from the multi-port valve back to the pool), was, if you can imagine, even worse. A barb stuck out of the multi-port valve, and attached to that was the 1 1/2″ vinyl tubing – using worm clamps. The tubing attached to another barbed fitting on the other side that was screwed into another enormous brass or iron valve that was so broken it was not ever used. The valve actually sat on a milk crate, and I never really had any reason (or time) to consider why this is: it’s because instead of the valve being attached to some kind of rigid pipe with an elbow, it just sat on a milk crate and was connected to the ground pipe using….. vacuum hose.

Yes, that vacuum hose. If you had a pool growing up, you might’ve swung lengths of this stuff around over your head to hear the noise it would make. Or you might’ve tied up your little brother with it. The point is, it’s a toy. It’s not meant for this kind of application. It’s the wrong size, but of course, worm clamps to the rescue!

I know this is all kinda hard to believe, so I took a few pics of the “before” version of the plumbing during the takedown/troubleshooting phase of my project.

IMG_1708.JPG

So above, Note that big valve – that’s coming from the pool into the pump. See all that black electrical tape? Yeah – that’s bad to see in a plumbing design. There was also some shrink wrap stuff on parts of the pipe that are under this table. You can also see here the infamous milk crate :-/

IMG_1709.JPG

Here’s the other half – vinyl tubing, barbs and worm clamps. Oh my! In its defense, it sorta worked, and had the nice side effect of being able to see the water moving through the tubing. I guess that’s cool. Of course, my pump has a clear lid so you can see what’s going on, and there’s that nifty pressure gauge on the multi-port valve so….

IMG_1713.JPG

This is a shot of my ‘testing’ setup. It’s a closed loop – the water comes from the bucket, up the hose on the left to the pump, and out of the pump back into the bucket. This worked wonderfully and allowed me to prove to myself that I didn’t need a new $x000 pump ;-)

The funnier part here is that you can now see clearly in the background that the return valve is connected (with a worm clamp!) to the black and white vacuum hose! :-P

I told you all of that so I could tell you this

Nobody ever questioned this monstrosity of a setup. Somebody came in and set this up, and somebody else paid for it, and was happy to not have to think about it. It was clearly shoved together with whatever parts the guy had lying around. And “shoved” is really the perfect word here. The bottom line is that this was a fiasco. An adventure. It was not taken seriously at all. I’m starting to see things like this more and more in both my professional and non-professional work. People don’t take the work they do seriously. They don’t think much about their work. They’re here today, gone tomorrow, and they’ll get paid on Friday, and that’s all that matters.

This whole plumbing system is just that – a system. Systems require thought. A well-functioning system is almost always the result of some amount of thought and design and perhaps (gasp!) discussion. Just because they’re locked away in a place nobody ever goes doesn’t mean they’re not important. In fact, most things that are put out of the way are put there specifically because they *are* important.

If you build systems, please take the building of those systems seriously. If you’re inheriting or coming into an existing system, or managing the building of a system, question it. Question everything. Ask why. Ask what the alternative solutions are. Ask what is compromised by doing it one way as opposed to another. Ask was is commonly done in your scenario and why. Be curious and skeptical. Look for red flags. Should $.10 tin worm clamps be used in a plumbing system? Won’t they rust to the point where they are unusable? Certainly this can’t be the optimal solution? What are some alternatives? Odd – I’ve never seen vacuum hose used in that way, can’t we use something else?

I know this is goofy, but this is how my brain works: every single thing you can learn is likely to be analogous to either a) something you already know or b) something you can more readily wrap your brain around. The good thing about that is that it means you can pretty much learn every single thing ;-)

In this case, my pool plumbing system is analogous to a computing system infrastructure, inasmuch as both are systems, made up of many and varied components – and each component represents a decision that must be made, and a potential source of a future problem. Does the component fit? Is it made fo this application? Does it integrate well into the rest of the system? I could be asking those questions of almost any component of almost any system whether it moves bits or water.

Well, enough rambling for one night. There was a message in that somewhere, but it’s late, and I’ve been working with PVC all day. ;-)

Two ssh-isms I forgot today

1. To remove an offending known_hosts key without opening the file or using sed or whatever, you can just run ssh-keygen -R <hostname>, which will remove any and all keys for the given hostname (see man ssh-keygen for other options like this).

2. To copy files from a local directory with spaces in its name to a remote one with spaces also in *its* name, escape the space in the destination directory *twice*. I was moving VMware virtual machines from one server to another today and it bit me. Here’s the right way to do it:

scp -P 2222 *.vmdk “localhost:/var/lib/vmware/Virtual\\ Machines/ks/.”

Note that I’m also working in an environment where everything I do requires various ssh tunnels and other workarounds, which necessitates the -P argument, and explains why it looks like I’m ssh’ing to localhost ;-)

(Thanks to deryni for reminding me about ssh-keygen – I was looking for an ssh flag for this)

UNIX mtime vs. ctime

Sometimes I get questions from people about stuff that I’ve long since taken for granted. One is “what’s the difference between mtime and ctime?”

The answer is simple, but I wanted to post it here in case it can help anyone. In UNIX:

mtime is “modification time”, and it is the time at which the last modification *TO THE CONTENT* of the file was made.

ctime is “change time”, and it is the time at which the last modification *TO THE PROPERTIES* of the file was made. By “properties” I mean things like ownership and permissions.  You can think of this as “chtime” – the last time a “ch*” command like chown, chmod, or chgrp was performed on the file.

New Job!

I started a new job about 6 weeks ago. I’m now doing infrastructure architecture at http://gfdl.noaa.gov

GFDL stands for Geophysical Fluid Dynamics Lab. It’s a NOAA site that supports atmospheric and climatology research. So in other words, the work I do supports research into things ranging from global warming to what the atmosphere on Mars is like to the weather here on Earth to simulations of the shape and movement of Katrina. I think of it as sort of an Institute for Advanced Study devoted to climatology research. Great minds in the field are here.

The research actually takes place at three different sites, DC, Boulder and Princeton, and affiliations with academic institutions flourish as well. In fact, I knew at least 4 people who worked here because of interactions between this site and cs.princeton.edu, my former employer.

My job, as it’s been described to me, is to provide a vision as to the design and direction of the infrastructure which supports the rather enormous high performance compute (HPC) cluster. This involves something of a learning curve to understand what’s here, how the systems are used, what the needs are, what people like and hate, where the redundancies and inefficiencies exist, etc. It also involves having meetings and coordinating with people who manage the network, the facilities (power & cooling, etc), the security policy, etc. I’ll be grilled on my ideas, and create prototypes and demos to get my ideas across. Lots of communication.

An aspect of my job will also involve getting my hands on the HPC clusters themselves as well, which are also at each site. All of the clusters are on top500.org last time I looked. Just go through the pages and search for GFDL and/or NOAA.

The systems here are all Linux. Even the standard-issue workstations are running Linux.  Scripting is done in Perl and shell, but Python is everywhere, so I’ll be doing either Perl or Python if I have the choice (because “shell” == “csh” here, which I never took well too, honestly). Some aspects of the environment are pretty fascinating. For example, how exactly do you store (*and* easily retrieve, on the fly) 9 PETABYTES of data? How do you back that up? How do you recover from hiccups? How do you instrument systems consisting of thousands of CPUs,  to pinpoint problems and get them fixed? And, by the way, how’s the best way to tune a system’s network stack to use a 50MBps pipe (that’s Mega *bytes*) efficiently enough to move multiple terabytes of data every day between collaborators at different sites? How, exactly, do you consolidate services and provide failover across geographically dispersed sites?

So that’s it for now :)  It’s too early to tell how things are going, really. It’s certainly not the cushy environment that Princeton U. was, but there are bigger challenges and problems to be solved here, and that’s the part I’m looking forward to.