Musings of an Anonymous Geek

Having a Google account is sometimes useful in ways you hadn’t planned for. For example, at a few different employers I’ve been at, I’ve had to prepare for reviews by providing a list of accomplishments to my supervisor. One decent tool for generating this list is email, though it can take some time. Another useful tool is the Web History feature of your Google account.

Though this isn’t necessarily indicative of everything I’ve accomplished in the first half of 2008 per se, it’s definitely indicative of the types of things I’ve generally been into so far this year, and it’s interesting to look back. What does your Web History say?

• Gearman - this is used by some rather large web sites, notably Digg. It reminds me a little of having Torque and Maui, but geared toward more general-purpose applications. In fact, it was never clear to me that PBS/Maui couldn’t actually do this, but I didn’t get far enough into Gearman to really say that authoritatively.
• How SimpleDB Differs from a Relational Database - Links off to some very useful takes on the “cloud” databases, which are truly fascinating creatures, but have a vastly different data management philosophy from the relational model we’re all used to.
• Reblog - I found this in the footer of someone’s blog post. It’s kinda neat, but to be honest, I think you can do similar stuff using the Flock browser.
• Google Finance APIs and Tools - did I ever mention that I had a Series 7 & 63 license two months after my 20th birthday? I love anything that I can think for very long periods of time about, where there’s lots and lots and LOTS of data to play with, where you can make correlations and answer questions nobody even thought to ask. Of course, soon after finding this page I found the actual Google Finance page, which answers an awful lot of potential questions. The stock screener is actually what I was looking to write myself, but with the data freely available, I’m sure it won’t be long before I find something else fun to do with it. I’m not a fan of Google’s “Feeds” model, but I’ve dealt with it before, and will do it again if it means getting at this data.
• Bitpusher - it was recommended to me as an alternative to traditional dedicated server hosting. Worth a look.
• S3 Firefox Organizer - This is a firefox plugin that provides an interface that looks a lot like an FTP GUI or something, but allows you to move files to and from “buckets” in Amazon’s S3 service.
• Boto - A python library for writing programs that interact with the various Amazon Web Services. It’s not particularly well-documented, and it has a few quirks, but it is useful.
• OmniGraffle - A Visio replacement for Apple OS X. I like it a lot better than Visio, actually. It has tons of contributed templates. You shouldn’t have any trouble making the switch. A little pricey, but I plunked down the cash, and have not been disappointed.
• The Python Queue Module according to Doug - Doug Hellmann’s Python Module of the Week (PyMOTW) should be published in dead tree form some day. I happen to have some code that could make better use of queuing if it were a) written in Python, and b) used the Queue module. I was a little put off by the fact that every single tutorial I found on this module assumed you wanted to use threading, which I actually don’t, because I’m not smart enough…. though the last person I told that to said something to the effect of “the fact that you believe that means you’re smart enough”. Heh.
• MySQL GROUP modifiers - turns out this isn’t what I needed for the problem I was trying to solve, but the “WITH ROLLUP” feature was new to me at the time I found it, and it’s kinda cool.
• Wordpress “Subscribe to Comments” plugin - Baron suggested that it would be good to have this, and I had honestly not even thought about it. But looking around, this is the only plugin of its kind that I found, and it’s only tested up to WP 2.3x, and I’m on 2.5x. This is precisely why I hate plugins (as an end user, anyway. Loghetti supports plugins)
• Lifeblogging - I had occasion to go back and flip through some of the volumes of journals I’ve kept since age 12, wondering if it might be time to digitize those in some form. I might digitize them, but they will *not* be public I don’t think. Way too embarrassing.
• ldapmodrdn - for a buddy who hasn’t yet found all of the openldap command line tools. You can’t use ‘ldapmodify’ (to my knowledge) to *rename* an entry.
• Django graphs - I haven’t yet tried this, because I’m still trying to learn Django in what little spare time I have, but it looks like there’s at least some effort towards this out there in the community. I have yet to see a newspaper that doesn’t have graphs *somewhere* (finance, sports, weather…), so I’m surprised Django doesn’t have something like this built-in.
• URL Decode UDF for MySQL - I’ve used this. It works really well.
• Erlang - hey, I’m game for anything. If I weren’t, I’d still be writing all of my code in Perl.
• The difference between %iowait in sar and %util in iostat - I use both tools, and wanted the clarification because I was writing some graphing code in Python (using Timeplot, which rocks, by the way), and stumbled upon the question. Google to the rescue!
• OSCON ‘08 - I’m going. Are you going? I’m also going to the Oregon Brewers Festival on the last day of OSCON, as I did in ‘06. Wonderful!
• Explosion at one of my hosting providers - didn’t affect me, but… wow!
• hypertable - *sigh* someday…when there’s time…
• Small-scale hydro power - Yeah, I’m kind of a DIYer at heart. I do some woodworking, all my own plumbing, painting, flooring, I brew my own beer, I cook, I collect rain in big barrels, power sprinklers using pool runoff to give my lawn a jumpstart in spring… that kind of stuff. One day I noticed water coming out of a downspout fast enough to leap over one of my rain barrels and thought there must be some way to harness that power. Sadly, there really isn’t, so I did some research. It’s non-trivial.
• You bet your garden - I also do my own gardening and related experiments.
• RightScale Demo - WATCH YOUR VOLUME - a screencast showing off RightScale’s features. Impressive considering the work it would take me, a lone admin, to set something like this up. The learning curve involved in effectively/efficiently managing/scaling/monitoring/troubleshooting EC2 is non-trivial.
• Homebrew Kegerator - Maybe if this startup is bought out I can actually afford this thing to put my homebrewed beer in. The 30-year-old spare fridge in the basement is getting a little… gamey.
• The pound proxy daemon - I use this. It works well enough, but I’ve crashed it under load, too. I’ve also had at least one hosting provider misconfigure it on my behalf, and I had to go and tell them how to fix it :-/
• Droid Sans Mono - a fantastic coding font. Installing this font is in my post-install routine for all of my desktops.
• Generator tricks for systems programmers - David Beazley has made available a lot of Python source code and presentation slides from what I imagine was a great talk (if you’re a systems guy, which I am).
• The Wide Finder Saga - I found this just as I was writing Loghetti. There are still some things in Mr. Lundh’s code that I haven’t implemented, but it was a fantastic lesson.
• Using gnu sort for IP addresses - I’ve used sort in a lot of different ways over the years… but not for IP addresses. This is a nice hack for pulling this off with sort, but it doesn’t scale very well when you have millions of them, due to the sort utility’s ‘divide and conquer’ method of sorting.
• Writing an Hadoop/MapReduce Program in Python - this got me over the hump.
• Notes on using EC2/S3 - This got me over some other small humps
• BeautifulSoup - found while searching for the canonical way to screen scrape with Python. I’d done it a million times in Perl, and you can do it with httplib and regex and stuff in Python if you want, but this way is at least a million times nicer.

Well, that’s a decent enough summary I guess. As you can see, I’ve been doing a good bit of Python scripting. Most of my code these days is written in Python instead of Perl, in part because I was given the choice, and in part because Python fits my brain and makes me want to write more code, to push myself more. I’ve also been dealing with things involving “cloud” computing and “scalability” — like Hadoop, and EC2/S3. I haven’t done as much testing of the Google utility computing services, but I’ve used their various APIs for some things.

So what’s in your history?

I’ve been working with what I used to call “utility computing” tools for about 6-9 months. However, for about the past 2 months, I’ve been seeing the term “cloud computing” all over the place, and there is so much buzz surrounding it that it’s reaching that magical point best described using Alan Greenspan’s words: “Irrational Exuberance”.

When Alan Greenspan used those words to describe the attitudes of investors toward the markets, what he was basically saying was that there were people who didn’t really know what they were doing, putting more money than they ought, into things they knew relatively little about. Further, he was saying that the decisions people were making with regards to where to put their money were a) bad, or at least b) not based on sound reasoning, or the ‘facts on the ground’.

This, I think, is where we are at with “cloud computing”. The blog post that put me over the edge is this one, for the record. I read Sean’s writings often enough, but this one strikes me as being a little off, a little sensationalistic, not based in reality, and a little misleading.

Maybe he just didn’t put enough qualifiers in there. His post might make more sense if he limited its scope and provided more facts, but I guess it’s just an opinion piece so he decided not to go that route, and that’s his prerogative I guess.

By limiting the scope, I mean he should’ve realized that there are millions of web sites currently scaling quite nicely without the use of cloud computing. In addition, some of the new ones that are having issues are also not using cloud computing, and when they hit bumps in the road, they make it through, and the great thing is that they also share their stories, and those stories indicate that a cloud (or, the current cloud offerings) wouldn’t have helped much (there’s lots of other evidence of that too). What would’ve helped is if they had paid more attention to:

• monitoring
• initial infrastructure design
• their own app code and app design

If there’s one thing that bothers me about using a ready-made solution like wordpress for my blog, it’s plug-ins. I hate software plug-ins. The first question every support engineer for any software product that supports plugins asks in response to a trouble report is “are you using any plugins?” And when you say “yep, I’m using plugins!” the reply from support is to disable them immediately and see if the trouble goes away. That’s a problem.

What’s worse, if the plugins are maintained by a third party (often the case), there’s no telling whether or not they’ll exist when the next version of the base software is released, or whether they’ll be supported in future versions of the software.

Two examples that touch my daily life are Firefox, and Wordpress.

Lately (since around March) I’ve been having lots of trouble with Firefox. I thought upgrading to Firefox 3 would’ve helped, but it really didn’t. Running it on OS X, Firefox hangs frequently enough that I’m actually considering using Safari (I do NOT like Safari). Know what happened right around that time? Ah - I found the firefox plugins for managing EC2 and S3. So today I’ll uninstall those and see if it helps.

With Wordpress, there are two things I’m missing: I need to let readers subscribe to comments via email, and I need better Google AdSense for Search integration with WordPress. Both things are kinda maybe supported in one version or another “but should work under…” - whatever. I don’t really want to spend my time downloading, reading the documentation to do the install, doing the install and configuration, etc., and then finding out that it doesn’t work, or worse, having it look on the surface like it works, but then finding later that it fails in evil-but-silent ways.

These two products are by no means exceptions. Moodle, PHP-Nuke, XOOPS, MediaWiki, Twiki, Postnuke… and for that matter, OpenLDAP, BIND, SSH, MySQL, Sendmail, PAM… all have plugins available written by other folks, and all have bitten me at one point or another. Usually when it comes time to upgrade the base software.

I’m not saying anything new here. People have had this problem with lots of different software products for a long time. My question is “why is this still a problem?” I’m not asking this because I have some magical obvious solution or answer, I’m asking because I feel like there’s probably more to it than I’m grasping. I’m not a masterful developer, or even a masterful software project manager, so I’m calling on all of you who are (or are closer than I am) to help me understand the problem. Some day, I might find myself in a position to take the wrong or right path where plug-ins are concerned, and I’d like to be more informed than I am so I can avoid putting users in the position I find myself in when I use other peoples’ software. Has Joel blogged this yet? If so, I can’t find it. Links please?

All right! In the past, some books seem to be delayed in getting into O’Reilly’s Safari site, but on the day that Baron announces the book’s arrival, I find that I’m able to access it in Safari right now! Sweet!

I’m going to OSCON in July, and I know that just about everyone I know who is a participant in this crazy life we call IT (or web 2.0, or whatever it’s called now), is flying to a conference or something in 2008. I’m starting to notice more and more posts like this one, so if you can avoid it, don’t put anything in a checked bag that you can’t afford to lose, and avoid US Airways, and pass it on, because when you see the list of things they don’t cover in their lost baggage policy, you’ll suddenly feel like you’re lucky to still have anything you ever checked with your bags.

I’m going to the O’Reilly Open Source Convention (OSCON) again this year. I went in 2006 as well, and had a blast, in addition to learning quite a bit, and meeting tons of people whom I’ve been acquainted with online for a long time. That was 2 years ago. Since then I’ve been acquainted with lots *more* people online, and I’m hoping I’ll meet at least some of them this year.

If you’re not going to OSCON, you’re not only missing out on a great technical conference that will leave you physically tired from all of the activity and at the same time unable to sleep from the ideas sparked by the day’s events, you’re also missing the Oregon Brewers Festival, which takes place just as OSCON is wrapping up.

I have a medium-sized home brewery that a buddy and I built from scratch. Over the years we’ve brewed and tasted all kinds of beer. But you can’t get all beers everywhere, so traveling is a good opportunity to taste wild and exotic beers, or just local beers you can’t get at home. It’s odd, but while you can get easy access to beers from Germany, Belgium, Poland, England, Scotland, and Ireland, you would be hard pressed to find a good number of great beers from the West Coast of the US on the East Coast of the US. And the West Coast has a lot happening, beer-wise.

Beer festivals are also where some brewers pull out all the stops. In ‘06 I went with a buddy and actually had not one, but TWO different watermelon beers - a variety I had not even heard of until I showed up at the counter. One was pretty good, the other tasted like Watermelon Bubblicious, but the experience was fantastic. Every have rock candy made from hops? Pretty good I tell you!

Anyway, I was thinking of getting a larger group together to attend this years Brew Fest, so if any geeks out there have an interest in beer, let me know. And if you DON’T have an interest in beer, you should DEFINITELY let me know. I’ve converted numerous friends and family who say they don’t like beer to becoming more familiar with styles they actually will go out and buy, unprovoked, voluntarily! Saying you don’t like beer is like saying you don’t like food. There’s just too many kinds of beer to say you don’t like beer. Maybe you don’t like hops, in which case you might like hefeweizen, but have probably never heard of it. Maybe you don’t like really fizzy beer, in which case you might like various Belgian ales, a Barleywine, a porter, or any beer with a less fizzy, more creamy, or less prevalent head on it.

Anyway, I’m going, and it’s fun. If you have an interest, do join in, whether you go with a group I put together or not!

A long time ago, a buddy of mine got me into brewing beer. I’m not talking about buying a can o’ syrup and adding water - I’m talking about buying grain, taking it home, milling it, mashing it, hopping it, fermenting it, bottling and kegging it…. *really* brewing beer.

Anyway, it turns out that a good number of the members of the local brew club in my area also work in IT. Since lots of other folks who visit here also work in IT, I figured I’d give a shout out to anyone who also brews, and point them at two things:

1. Some time ago, I started an IRC channel for home brewers. The channel is #homebrew on irc.freenode.net - please join us!
2. I’ve been the primary poster on the blog my buddy and I started to post our recipes and notes and stuff. You might find some useful stuff there http://www.bamfbeer.com

Blogged with Flock

I came across this entry on autobloggreen.com about the Chevy Volt, which is GM’s latest stab at convincing us that they really do put some effort behind getting an electric car out the door to us sometime in the next millenium. I have to admit that, while I’m wholly *unconvinced* that I’ll ever see this car on a showroom floor near me, this car is totally cool.

My disclaimer is that I have clearly not done the research to figure out if this car is as green as it could possibly be or whatever; I don’t know if it takes more energy than traditional means through the charging process, mainly because I’m not an engineer. But the concepts they’re working with and the problems they’re aiming to solve at least shows that they understand that simply putting out a car that runs on electricity and forgetting about every single other detail will not fly.

First of all, this car isn’t something you would shudder to be seen in. Sure, it’s no Ferarri, but neither is anything else (except… a Ferarri). This car looks worlds better than those cheeseball little fiberglass boxes they wrap around the hybrid vehicles.

Speaking of hybrid, I’ve always had a problem with hybrids, and the Chevy Volt improves upon one aspect of the hybrid that I dislike: I’ve always looked at the design of the hybrids as a sign that the technology is not done yet. If it were, you wouldn’t need it to be “hybrid”. Let’s not forget that “hybrid” in “hybrid vehicle” is referring to the fact that there are two technologies working to get the job done, because one of them is costly, wastes energy and gives off emissions (that’d be the gas) and the other one nobody has learned to implement in such a way that it can replace the first one.

Well, GM didn’t totally throw gasoline out the window, but they’re using it in a bit of a different way from traditional hybrids - here’s the deal: you plug the Volt in to charge, say, overnight. In the morning, it has a full charge, which is enough to get most people back and forth to work (they’re shooting for a 40-mile range). However, it also has an internal combustion engine capable of running on gasoline or any of a number of bio-fuels. This engine isn’t connected to the wheels in any way, but rather the engine is used to charge the batteries. In this configuration, a car with a full tank of gas (a 12-gallon tank) and a full charge will go 640 miles. For reference, this is a bit more than double the distance I could go in my old Chevy Celebrity, which I believe had a 16-gallon gas tank.

So now the issues. As usual, the battery technology isn’t done yet, and when they’re done with that, they need to figure out how to make the thing cheap enough that people will actually buy the thing. Both issues seem to be major unknowns right now. I wish them luck, because I’d like to have something like this sooner than later.

Technorati Tags: auto+industry, green+technology, chevy+volt, hybrid, fossil+fuel, opinion, electric+car