You Might Be a Freelancer…

  • If you’ve ever used your lunch hour to vacuum your pool…
  • If you’ve ever discounted your rate to work on something really cool…
  • If your spouse has ever complained about the mountain of receipts in the basement…
  • If you work 8 hours a day… and 3-5 hours a night…
  • If you’ve ever spent an entire day shopping for that perfect pair of flip-flops…
  • If you fold laundry during conference calls…
  • If you own as many books about “Consulting Success” and Accounting as you do about your area of expertise…
  • If you’ve ever had to tell your dog to get out of your “cubicle”…
  • If you’re able to accurately rate the quality of free wifi at any place within a 10-mile radius…
  • If you’ve ever tried to see just how many client projects you could tackle at once without your brain exploding or divorce proceedings…

…You Might Be a Freelancer.

LinuxLaboratory.org Django Update

Hi all,

So, I’ve begun down the road of reinventing LinuxLaboratory.org using Django. For the moment, I’m mostly starting out using the blog application from the newly released second edition of “Practical Django Projects”, and there’s no CSS or fancy design going on yet, but I have a nice admin interface for creating posts, managing categories and tags, etc., and I’ve created some basic templates that do enough that I can see things working.

I will eventually add all of the old content (mostly system administration and scripting articles I wrote for O’Reilly, Linux.com, and others over the years) back into the site, but I started with the blog so that I could share my progress and (when I enable comments) get feedback and tips as I move along.

There’s no RSS feed yet, either, but that’s coming as well. You can follow along for the moment by just going to the site, or I’ll post short summaries of what’s going on here.

Awk Idioms: Shorten your pipelines, consolidate your tool set

I was lurking around on twitter during my lunch hour (yes, even freelancers need a lunch hour), and @bitprophet tweeted thusly:

Get syslog-owned log names from syslog.conf: grep -v “^#” syslog.conf | awk ‘{print $2}’ | egrep -v “^(\*|\|)” | sed “/^$/ d” | sed “s/^-//”

Followed by this:

(Interested to see if anyone can shorten my previous tweet’s command line, outside of using ‘cut’ instead of the awk bit.)

I happen to love puzzles like this, and my lunch was almost immediately followed by a long, boring conference call.

@bitprophet’s pipeline above is translated by my brain into the English:

Find non-commented lines, grab the second space-delimited field, then filter out the ones that start with “*” or “|”, then delete any blank lines, and strip any leading “-” from the result.

My brain usually attempts to think of the English version of the solution *first*, and then try to emulate that in the code/command I write. So, the issue here is we want to find file paths (and apparently sockets are ok, too, as “@” is a valid leading character in the initial definition of the problem). If it’s a file path, we want to see it in a form that would be suitable for passing it to something like “ls -l”, which means leading symbols like “-” and “|” should be omitted.

In a syslog.conf file, the main meat is the area where you specify the warning levels, and the file you want messages at that warning level sent to (this is a simplistic explanation, but good enough to understand the solution I came up with). The file is also littered with comments. Here’s the file on my Mac:

*.err;kern.*;auth.notice;authpriv,remoteauth,install.none;mail.crit        /dev/console
*.notice;authpriv,remoteauth,ftp,install.none;kern.debug;mail.crit    /var/log/system.log

# Send messages normally sent to the console also to the serial port.
# To stop messages from being sent out the serial port, comment out this line.
#*.err;kern.*;auth.notice;authpriv,remoteauth.none;mail.crit        /dev/tty.serial

# The authpriv log file should be restricted access; these
# messages shouldn't go to terminals or publically-readable
# files.
auth.info;authpriv.*;remoteauth.crit            /var/log/secure.log

lpr.info                        /var/log/lpr.log
mail.*                            /var/log/mail.log
ftp.*                            /var/log/ftp.log

install.*                        /var/log/install.log
install.*                        @127.0.0.1:32376
local0.*                        /var/log/appfirewall.log
local1.*                        /var/log/ipfw.log
stuff.*                            -/boo
things.*                        |/var/log
*.emerg                            *

So, in English, my brain parses the problem like this:

Skip blank lines, commented lines, and lines where the file name is “*”, and give me everything else, but strip off characters “-” and “|” before sending it to the screen.

And here’s my awk one-liner for doing that:

awk '$0 !~ /^$|^#/ && $2 !~ /^\*/ {sub(/^-|^\|/,"",$2);print $2}' syslog.conf

Knowing a few key things about awk will help parse the above:

Awk automatically breaks up each line of input into fields. If you don’t tell it what to use as a delimiter, it’ll just use any number of spaces as the delimiter. If you have a CSV file, you’d likely use “awk -F,” to tell awk to use a comma. For /etc/passwd, use “awk -F:”. From there, you can reference the first field as $1, the second as $2, etc. $0 represents the whole line. There are more, but that’s enough for this example.

Though I think most sysadmins can get a lot done with simple usage like “awk -F: ‘{print $2}’”, sometimes more power is needed, and awk delivers. It uses the basic regex engine, and enables you to check a field (or the whole line: $0, like I do above) against a regex as a precondition for performing some action with the line or a field on that line. So, in the above awk command, I check to see if the line is either empty, or a comment. I then use a logical AND to check if field 2 starts with “*”. If the current line is a match for any of these rules it is skipped.

Another nice thing about awk is that it actually is a Turing-complete programming language. After I check the lines of input against the rules mentioned above, I immediately know that I definitely want at least some portion of $2 in the remaining lines. What I *don’t* want are preceding characters like “-” or “|”. I need to strip them from the file name. I use awk’s built in “sub()” function to handle that, and with that out of the way I call “print” to send the result to the screen.


	

LinuxLaboratory woes, Drupal -> Django?

Ugh…

So, today I tried browsing to one of my sites, linuxlaboratory.org, and found a 403 “Forbidden” error. Calling support, they said it was a “billing issue”. Well, I pay my bills, and I haven’t received any new credit cards, so I’m not sure what that’s about. Further, they haven’t contacted me in any way shape or form at all in a very long time, and I’ve had the same email addresses for years now. Last time they failed to contact me, it was because they were sending all of the mail to “root@localhost” on the web server.

What’s more, the tech support guy, having determined that this wasn’t a technical but an administrative problem, transferred me to a sales person who was not there. I left a message. That was 3 hours ago. So I took matters into my own hands and changed the name server records to my webfaction account, and linuxlaboratory.org now points to an old test version of the site that uses Drupal.

It’s Over Between Us…

Drupal holds the record for the CMS that has run LinuxLaboratory the longest. Since its launch in 2001, LinuxLaboratory has used all of the major, and some of the minor open source PHP CMSes. Drupal gave me something very close to what I wanted, out of the box. Nowadays, Drupal is even nicer since they redid some of the back end APIs and attracted theme and module developers to the project. I’ve even done some coding in Drupal myself, and have to say that it really is a breeze.

But the problem is this: I’m a consultant, trainer, and author/editor. I am an experienced system admin, database admin, and infrastructure architect who makes a living solving other peoples’ problems. I really can’t afford to have something that is super high overhead to maintain running my sites. With Drupal releasing new versions with major security fixes once per month on average, and no automated update mechanism (and no built-in automated backup either), it becomes pretty cumbersome just to keep it updated.

This is in addition to my experiences trying to do e-commerce with Drupal. I tried to use one plugin, but soon found myself in dependency hell — a situation I’m not used to being in unless I’m on a command line somewhere. So, out with Drupal. I know it well and I’m sure I’ll find a use for it somewhere in my travels, but not now, and not for this.

Is Django the Future of LinuxLaboratory?

So I’m thinking of giving Django another shot. In fact, I thought I might try something new and interesting. Maybe I’ll build my Django app right in front of everyone, so that anyone who is interested can follow along, and so people can give me feedback and tips along the way. It also lets me share with people who have questions about a feature I’m implementing or something like that.

For fanboys of <insert technology here>, know this: I’m a technology whore. I consume technology like some people consume oxygen. I love technology, and I get on kicks, and every now and then, a “kick” turns into a more permanent part of my tool chest. Python is one such example. I’ve done lots with Python, but have never really made friends with it for web development. I got a webfaction account specifically because they support Python (and Django). I’ve done nothing with it. Now I think I might.

But not to worry! I own lots of domains that are sitting idle right now, and I’m considering doing a Ruby on Rails app for one of them, and I’m dying to do more with Lua. There’s only so much time!

Webfaction Django Users: Advice Hereby Solicited

So if you’re a webfaction customer using Django, please share your tips with me about the best way to deploy it. I’ve used nothing but PHP apps so far, and found that rather than use the one-click installs webfaction provides, it’s a lot easier to just choose the generic “CGI/PHP” app type and install the code myself. This allows me to, for example, install and update wordpress using SVN. Is Django a similar story, or does webfaction actually have an auto-upgrade mechanism for this? How are you keeping Django up to date?

Thanks!

I’m Offering Pro-Bono Consulting

I started my company about a year ago, but I’ve been doing consulting for a long time. In fact, my first job in the IT industry was working for a consulting firm. Before that, starting as far back as grade school, I was involved in a lot of volunteer civic and community service activities. I admire companies who get involved in their communities, or even outside of their communities, wherever help is needed.

As part of my business plan, I’ve put in place a policy of accepting one pro-bono consulting project per year. So far, I haven’t gotten any requests for free consulting work, so here’s my public shout out to let you know what types of services are available:

1. Speaking or Training. My specialties are things like advanced Linux administration and SQL, but I’m perfectly capable of delivering content for people who just need to know how the internet works, or want to know more about social media.Training, funny enough, has been the bulk of my business for the past year.

2. I can help with MySQL performance tuning on *nix systems, including finding hotspots related to the design of the database itself, or how your application code interacts with the database. If it happens that your MySQL server is performing poorly due to an underpowered system, I can also pinpoint which resource is dragging on the performance of your database.

3. If you just need random scripts written to perform *nix system administration tasks, I can consult with you about the requirements and write them for you. Note that while I can script in several languages, my preference for anything longer than 40 lines of code is Python.

4. I can build PC’s, install networks, set up firewalls and wireless routers, and all of the normal “office IT” functions, but note that my consulting is Linux consulting. I don’t work with Windows (well, I do, but not for free) ;-)

5. If there’s some other thing you’ve seen me blog about here, chances are I’ll be willing to perform a pro-bono consulting engagement to do it for you, or show you how to approach a problem, a large project, a migration, automation, monitoring, security or whatever.

Unless you happen to live within commuting distance to Princeton, NJ, work will be done remotely :)

Please email your request to jonesy at owladvisors dot com. Include your organization’s name, your contact info, and as much detail about the project and what your organization does as possible. The decision of which project to take on will be based solely on the information in your request!

Training Patterns

So, I’ve been talking to some friends about training. I work with several firms in various aspects of training. Usually I’m actually delivering training, but in some cases I’m just helping to produce the training content, and occasionally I actually help build a full-fledged internal training regimen. I love doing this work, because it lets me draw upon my work experiences going all the way back to age 14!

No, seriously. Most jobs have some form of training, and even if that training results in you doing some relatively menial job like being a waiter or a shipping clerk, it’s still training, and at a high level, the idea behind that training isn’t much different from the technology training I deliver for my clients. I have also gotten some fantastic ideas about how training can be implemented by remembering the training I received, or talking to friends who worked with me, or who worked at other jobs at the time which I assume had some form of training.

There are a couple of interesting patterns that pop up a lot in training that I have not seen implemented in IT shops that I think could work fantastically well. At some point, I’ll write about them (is that a book idea? Would you read it? Let me know), but for now, I actually want to hear from YOU!

Have you ever received training so good that you felt as qualified as the trainer himself to do the job at hand? Do you remember a training experience that had you seeing light bulbs seemingly every five minutes? Is there training you received that was so good that you still remember it over 20 years (or $long_time) later? I’d really like to hear about it. If you don’t want to put it in the comments, please send it to me via email at bkjones at gmail dot com.