Comments (8)

By m0j0, June 17, 2009 10:34 am

I have a need for this hack every now and then, and I *always* forget it for some reason, so I’m putting it here for safe keeping. I’ll start a “hacks” category, too, so I can locate these quickly, and so you can too So, here’s the hack:

[jonesy@cranford testing]$ ls
bar  foo
[jonesy@cranford testing]$ ls foo
1  2  3
[jonesy@cranford testing]$ ls bar
1  2  3  4  5  6
[jonesy@cranford testing]$ yes n | cp -i bar/* foo 2>/dev/null
[jonesy@cranford testing]$ ls foo
1  2  3  4  5  6
[jonesy@cranford testing]$ ls bar
1  2  3  4  5  6

I’m piping a constant “no” to the “cp -i” command, which asks you if you want to overwrite (or not) any files that already exist in the target directory. You don’t have to send STDERR to /dev/null — there’s just some messy output if you don’t. The end result is that the copy command will only copy files into the destination directory if they don’t already exist. It’ll skip files that exist in both directories (Well, technically the *copy* command won’t, but the operation will when you do it this way).

Of course, I could just forcibly overwrite the directory contents, but I don’t know if the files that exist in both directories are identical. Or I could move one sideways and the other into place, but the same issue exists.

Comments (0)

By m0j0, June 12, 2009 2:23 pm

So, I’ve taught the Linux Boot Process as part of a couple of different training courses now, and I thought I’d share it with the world in the form of a screencast (it’s hosted at my co.’s site). This is also a test to help me figure out how to “do screencasts”, generally.

The material in the screencast is slightly adjusted, because different training clients want to see different things, and some just can’t afford to spend a lot of time on the boot process in their training classes.

I welcome any feedback anyone might have on it.

Comments (7)

By m0j0, June 3, 2009 9:08 am

So, someone showed me a script they were writing in bash. They were doing a lot of manual menu creation, then using “read” to collect input, and manually mapping the input to longer strings, and then doing string manipulation stuff. Trying to help simplify and make things a little more sane and readable, I discovered that I was going to break some of my own rules with regards to bash scripting:

1. It was likely going to involve arrays, which is usually the time I migrate to Perl or Python.
2. It was going to be longer than 60 lines, which is usually when I migrate to Perl or Python.
3. The script didn’t make calls to any standard system commands, which means bash was being used for its generic scripting structures like case statements and if-test checks, which is something I try to foresee when I write scripts. If I’m not greatly aided by standard CLI utilities, I try not to do it in bash, because in the end, it’s a shell, not a scripting language.
4. There was no requirement to use bash, and Python and Perl are both available.
5. XML was involved.

The first order of business, though, was to replace all of the menu-related stuff with “select” loops instead of doing all of the manual stuff. It made things a lot simpler. Once that was out of the way, I inspected more of the code, and at that time realized “oh wait a minute….”, and became slightly ill as the above menu items flashed into my brain.

So I set about the task of fooling around with the task in Python. My first discovery there was that there isn’t any “select”-like functionality built into Python. Then I tried the curses module, but that makes doing “select”-like menus even messier, because not only does it *not* store the *string* associated with an option, it doesn’t even store the option string directly — it stores the ASCII code. This makes curses a lot more robust than using bash, but between messing with screen, window, and box semantics, and then still having to do some kind of option-to-value mapping by hand, it’s not much of a win in the simplicity department.

There are a couple of third-party modules that deal specifically with text-based, menu-driven, non-curses apps, so solutions do exist (this one looks good at first glance), but it requires altering a stock, rpm-based Python installation, which isn’t so bad except that it also means maintaining that alteration going forward — something I’d like to avoid.

What ’select’ Actually Does in Bash

Here’s a bit from the bash reference manual, which is way more concise than I’m ever likely to be:

select

The select construct allows the easy generation of menus. It has almost the same syntax as the for command:

          select name [in words ...]; do commands; done

The list of words following in is expanded, generating a list of items. The set of expanded words is printed on the standard error output stream, each preceded by a number. If the ‘in words’ is omitted, the positional parameters are printed, as if ‘in “$@”’ had been specified. The PS3 prompt is then displayed and a line is read from the standard input. If the line consists of a number corresponding to one of the displayed words, then the value of name is set to that word. If the line is empty, the words and prompt are displayed again. If EOF is read, the select command completes. Any other value read causes name to be set to null. The line read is saved in the variable REPLY.

The commands are executed after each selection until a break command is executed, at which point the select command completes.

Here is an example that allows the user to pick a filename from the current directory, and displays the name and index of the file selected.

          select fname in *;
          do
          	echo you picked $fname \($REPLY\)
          	break;
          done

I’ll figure out a solution that isn’t too ugly and post it here. In the meantime, if you’ve already done all of that, or I missed something in the docs for Python or curses, post links or code in the comments. Thanks!

PS – Before I get flamed: I chose Python because it’s my go-to language for all of my scripting tasks. I don’t recall Perl having an easy way to do this either without also installing an external module, and I figure I’ll learn something new about my favorite language while trying to solve this problem.

[UPDATE] – As soon as I launched the Python interpreter and started thinking along the lines of Python dictionaries, things fell into place:

>>> d = {1:”foo”, 2:”bar”, 3:”baz”}
>>> d
{1: ‘foo’, 2: ‘bar’, 3: ‘baz’}
>>> for k,v in d.iteritems():
…     print k,v
…
1 foo
2 bar
3 baz
>>> d.get(1)
‘foo’

That works, is still much cleaner than bash “read”, and gets me the option-to-value mapping, without loading *any* modules.

[UPDATE 2: It was pointed out that I failed to explain what 'select' actually does, so I've added a description from the bash documentation, which I think explains it pretty clearly. ]

Comments (2)

By m0j0, May 26, 2009 10:01 pm

So, I’ve been freelancing now for 9 months. I did a post a while back about what was working and what wasn’t, and I still stand by those recommendations. But that was over 6 months ago. Since then a lot of things have happened. I’m happy to report that, so far, things are going great. My clients are happy, I’m happy, and I’m taking to the business end of things pretty well.

However, I’m learning more about the business, and myself, so I thought I’d share s’more thoughts on my freelancing experience to help those in the same boat, or who are thinking of making the leap:

You only *think* you’re organized

I thought I was pretty organized before I became a freelancer. I was always punctual, never missed appointments or meetings, didn’t miss deadlines — I was on top of things. I’m still doing all of that, but I’m finding that I have to organize things very differently from when I was a 9-to-5 employee.

This might not apply to you. You might already organize things in a way that would be easy to apply to a freelancing schedule. It wasn’t that way for me. I typically used to think of my life in terms of projects. I had projects I was working on, usually more than one at a time. Every week, I had a meeting with my boss to go over the status of the projects, if I had hit any roadblocks, etc. In retrospect, it was pretty nice.

Nowadays, I still have projects, but they all belong to different clients, and all of the clients need their own status, and they all have different personalities, different lingo, different businesses, and different priorities. Some client projects are heavily focused on a completion date. Others are heavily focused on a feature set. Still others are focused on a budget number. I have to interact with them all separately, and I have to think about how to present the status to each one to meet their expectations. So, I actually budget time to think about these things.

What’s more, since I do all of my work remotely on an hourly basis, if a client has to complete a task so that I can move forward, and it takes them three days, I may well have nothing I can work on and get paid for for that period of time. With some clients, this isn’t a very big deal: I offer them a discounted rate if they prepay for “bulk hours”, so they get a cheaper rate, and I’m not out of luck if there are delays. I also try to work in other smaller projects and do business development work for my company during these times.

So, the lesson is not to bother trying to come up with crazy Gantt charts and precise time lines. Your clients’ businesses don’t work that way, so yours will need to be flexible as well.

Nowadays I create more loosely defined, high-level project definitions with less detailed tasks and time lines. I still don’t lose sight of what needs to be done, and I’m still able to meet my clients’ needs.

I’m more paranoid than I thought

I’ve grown used to people asking me “How’s business?” I used to hate that question, because I didn’t really have a metric to go by that I thought was sufficient. Now I have several metrics, but I feel like I need to evaluate where I am with respect to those metrics almost daily. I guess it’s part of being a young business.

When you’re fully employed, “business”, to some degree, is always good, because you’re always employed, and always have a paycheck coming in. My stepfather worked for Exxon during the Valdez oil spill. Business for Exxon was horrible. But my stepfather’s job (i.e. “business”) was relatively unaffected.

My business has two services: consulting and training. I try to maintain training appointments for at least the next three months, and consulting projects for the next 4-5 months. If I look at my calendar and see lots of empty spots any time in the next three months, I have work to do. This methodology works for me, and makes me less stressed out than the method I used to use.

I used to just say “if at any time I have less than 6 months of expenses in the bank, I have work to do”, but looking at dollar figures every day is stressful, and I think in terms of the business it’s kinda like taking your eye off the ball. The ball isn’t money — the ball is your client list. Much like the ball brings with it the potential for a base hit or home run, your client list has the potential to pay your bills and grow your business.

More to Come

I can get a little long-winded, so I’ll stop here for now. I’m interested in your input, whether you’re a freelancer, or considering becoming one. Share your thoughts!

Comments (0)

By m0j0, May 19, 2009 12:05 pm

First, I want to point out the only thing I hate about this application: the name. It’s bad for me, it’s bad for the developers… it’s just bad. If I ever had a problem, how am I supposed to do a search for that? If the developers want to find blog posts like this one, how are they supposed to do that?

Aside from the name, Things for Mac really, truly shines. I’ve tried lots and lots and lots of task managers, project managers, life managers, blah, blah, blah. I’ve probably tried 15 of them, and really tried hard to like them, but I just couldn’t. Some are too rigid and structured, while others are completely lacking any structure whatsoever. Some have a learning curve that forces you to make time to sit down and learn them. Others take pains to pay special attention to every aspect of your life (a model doomed to failure). I don’t want to mention all of the ones I’ve tried, because they’re not bad products and don’t deserve bad press. Mostly, they all worked well — they just were not right for me.

Things, on the other hand, I like. It’s just the right balance of features, structure, and ease-of-use. After watching the screencast introduction, I was off and running, and haven’t looked back. I wrote a short poem (a la Dr. Seuss) about my experience:

I do not like them on my Dock,
I do not like them in Firefox.
I do not like them in my car,
I do not like them near or far!
I do not like most “task”ish apps,
but I find I like this “Things” for Mac!

Ok, it’s totally corny. On to what I like about it.

On the Desktop

I actually didn’t expect to like the idea of a desktop application. Of the 15 other tools I’ve tried, only one or two others had a desktop app, and those two apps were opposite ends of the spectrum: one required a PhD to use, and the other was more like a free-form note pad. Things is neither. If you’re in a rush and just want to make sure you don’t forget a task, you can click on ‘Inbox’, type in as many or as few task details as you want, and be done with it. Later, you can drag the task into a project, or add scheduling information, make it repeat, etc.

Even better, it has a keyboard shortcut that’s settable, so if the app is running somewhere, you can just use that to do a “Quick Entry” without leaving the app you’re currently using. Very handy.

Though Things is not project management software, you can create “Areas of Responsibility”, and then have projects inside of those areas. This is perfect for me. I have areas of responsibility like “Home”, “Work”, and “Blog”, and inside the “Home” area I have all of my home improvement projects. My “Work” area contains a list of projects, which themselves contain high-level tasks (I leave the detailed stuff to actual project management software), and also projects which are related to business development, which I don’t use project management software for. The “Blog” area is a perfect place to jot down ideas for blog posts that I can go back to later and check off when they’re completed.

You don’t have to worry too much about being reminded about upcoming due dates, assuming you set due dates (which you don’t have to do). When due dates are approaching, Things will move the tasks into the “Today” queue, so if you have some down time during the day and aren’t sure what to tackle next, click “Today”, and start on something! I find that there’s almost always something in that queue that I have time to do before I need to (or am able to) get back to what I was doing. That, alone, is worth the $50 I paid for Things.

On the Go!

I’m not sure I would be so into Things if it weren’t for the accompanying iPhone app, which pretty much rocks. I guess if that didn’t exist, they would’ve considered having a companion web interface or something, but really, given the choice, I’d rather have it be mobile than solely on the web (though I’d like to see Things work on non-Apple mobile devices too).

One of the really awesome things about the iPhone app is that it syncs over the air to the desktop app. Of course, both have to be running, and they need to be on the same network, but my desktop app is *always* open, and I have yet to be inconvenienced by anything related to the sync functionality. Some folks have requested mac-to-mac sync capability, which isn’t currently there, but I don’t need it, so it’s not an issue for me.

While the iPhone app is missing some features of the desktop application, 90% of what’s in the desktop app is doable from the iPhone. The only notable exception for me personally was an apparent inability to create new areas of responsibility. Indeed, this is the one place where the interface is a little inconsistent. While clicking “Inbox” or “Today” will bring you to a list of tasks and projects, they chose to list projects and areas of responsibility in the sidebar for easy perusal and drill-down capabilities. It works well in the desktop app, but it’d be nice to make “areas” a queue on the iPhone, just like “Projects” is.

A Compeling Duo

In the end, Things has two things going for it: First, it has the nicest, easiest to use interface. Second, it has the ability to sync over the air with my iPhone, which also has a nice interface. In the old days, you might leave your bulky planner at your desk and carry around a note pad, and this mimicks that kind of experience, but takes away the need to sync them up later by hand.

It’s simple, but not too simple. Structured, but not too structured. I’ve been using it for a little over 2 weeks now, and have no plans to change. I know this sounds like an ad, so I should also say that I’m not being paid for this and don’t work for the company that makes it. I do recommend you check it out, though, if you’re in the market for something to help manage your time.

Comments (0)

By m0j0,

So, my training business is doing better than expected, which is good news in a tough economy.

I’ve gotten some feedback from readers about my mention of training services offered through my company. My company has historically only performed training at client sites, and some folks had hoped I would expand into offering “open enrollment” courses at some point. Of course, they’d like it if I did that all over the country, but I’m only one guy (for the moment).

What I *have* done is I’ve set up an open enrollment course not too far from where I live. If you’re anywhere near Edison, NJ and need a beginner’s course in Linux administration (not *using* Linux — *administering* Linux), then you can either contact me directly to see how to reserve your spot for either the August 3-7 or October 5-9 course, or you can wait for the online order link to appear on this page.

Why Open Enrollment?

There are two main reasons people wanted an open enrollment course, and I’m trying to respond to those people who gave me this feedback by offering this course. The two main reasons are:

1. It’s cost prohibitive to do on-site training unless you have ~5 or more people.
2. Some companies don’t have space or equipment to host on-site training.

If you only have three people to train, on-site training is probably going to cost more than sending your people off to a training center. However, I would still suggest on-site training for the following reasons:

First, if you only have three people to train, chances are it’s going to be painful to have them out of the office all at once for an entire 5-day training course.

Second, if your company is somewhere between Philadelphia and New York City (commuting distance from Princeton, NJ), local companies get a discount to bring the cost of on-site training down.

As for the problem of not having training space, that can be harder to overcome, no matter how many people need training. If your company needs training outside the NY/NJ/PA area, and doesn’t have space to do it on-site, let me know. I have a growing collection of companies in this situation. At some point, I can group the students into a training course and try to find lab space to hold the course in your town.

Comments (0)

By m0j0, April 29, 2009 7:26 am

Everyone please also point this article at the managers who purchase training for your company or team, and other people you know who’ve had Linux training.

So, a very large part of my business is providing Linux training. Up to now, I have performed only on-site training, often times developing custom training content for clients who have needs that aren’t met by “off the shelf” training. Often times, this has been extremely advanced training, though I also had a client who couldn’t find training that was geared toward people who had almost no exposure to Linux, and only basic Windows skills.

Now, I’m planning to begin offering off-site training at lab facilities, just like the big training giants. The difference being that I have the benefit of having the agility and flexibility of being a small company that can react with lightning speed to my clients needs. If you can and would like to help me out in shaping my training offering, I’d love to hear your input as to what’s missing from the Linux training landscape.

Here are a few questions I’m wondering about, but if you have other insights, those are also welcome. If you don’t want to answer in public, please shoot me an email to jonesy at owladvisors dot com.

• Have you had some Linux training lately that fell short of your expectations?
• What content was missing? What bad assumptions were made about the students?
• What was missing from the training web sites themselves? Is it hard to purchase?
• Are you looking for group discounts?
• What kind of information are you looking for when you buy that you’re not getting?
• Did the materials given out in the class fall short in some way? What were you expecting to receive?
• Did the instructor fall short in some way? What is the ideal instructor in your eyes?
• Do you purchase training solely for certification, or are you more interested in enriching your skill set?
• What kinds of things are you doing with Linux that you can’t find any training for? Be as specific about tools/libraries involved as you can without getting in trouble
• What’s the most annoying experience you’ve had in looking for or being a student in a Linux training course?
• What (non-commercial) tools or techniques do you use every day that are strictly “tribal knowledge”: you’re sure a Linux training class would never cover them in a million years.

Thanks everyone for helping out! Also, if you have any questions for me about training, I’m happy to answer those as well. Post in the comments (I’ll reply!) or email jonesy at owladvisors dot com.

Comments (1)

By m0j0, March 30, 2009 8:35 pm

To all my geek/nerd friends in the blogosphere: I’ll be posting updates on Fedora Directory Server, my Linux training courses, and more in the coming weeks, but I wanted to let you know that I’ve recently been stricken with… umm… Twitter. I’m @bkjones on twitter, so if you’re into beer, brewing, billiards, cooking, guitar/music, linux, system administration, perl, shell, python, php, databases, sql, or anything like that, lemme know, or follow me!

Comments (0)

By m0j0, March 25, 2009 8:52 am

I know, I know. I haven’t been posting nearly enough. But I did come across two URLs that are too handy not to pass on:

1. Command-line-fu: this is a repository of handy one-liners submitted by pretty much anyone. You can log in with OpenID or register on the site itself. I expect this, or something like it, will become a great resource. You can browse the sweet one-liner goodness, or “grep the archive”. Nice.
2. Down for Everyone or Just me? is a site that’s handy to know about if you’re, say, holed up in a hotel room, forgot to set up port forwarding on your FIOS router, and so don’t have a remote shell to test from, and you can’t reach a site. Pop the url into this site, and it’ll test access for you. Of course, it’s limited — it’ll change url’s with “:22″ to “%3A22″, so you’re not going to get it to be a generic service tester, but still… handy!

Enjoy!

Comments (0)

By m0j0, March 11, 2009 10:37 pm

The last time I had to do a NIS->LDAP migration, it was in a heterogenous environment with Solaris and Linux boxes, and it was around 2004 or so. Although I hit some rough patches adjusting to changes in how FDS is packaged, the community was awesome, and helped me get back up to speed in no time. We shouldn’t forget that the community was what drove me from OpenLDAP to FDS in the first place.

But I digress. The purpose of this article (first of a series) is to share with you some technical information about how to get things going. How, exactly, do you get RHEL 4, and RHEL 5 to utilize Fedora Directory Server’s data to support NSS and PAM for user information and authentication, and autofs for automounting directories? There are documents on this, written by people who clearly do (or did) care, but at times they can be a little disjointed, a little outdated, and require some tweaking.

This document talks specifically about installing the fedora-ds-1.1.2-1.fc6 package on RHEL 5.2, populating the People and Groups trees, and testing that it actually works. Later posts will deal with getting RHEL 4 and 5 clients to talk to it for various purposes, using TLS (with certificate verification, btw).

If your real issue is understanding how LDAP data works, why it looks the way it does, or you need a refresher, I would urge you to look at two other articles I wrote for O’Reilly, devoted completely to the topic: here, and here.

Get it installed

There is no precompiled binary package of Fedora Directory Server built specifically for Red Hat
Enterprise Server (because Red Hat, of course, provides that, with support, for a fee). If you want to run FDS for free on a RHEL server, the installation process is somewhat non-trivial.  First, you must add a couple of new package repositories to your yum configuration:

cd /etc/yum.repos.d/
sudo wget http://directory.fedoraproject.org/sources/idmcommon.repo
sudo wget http://directory.fedoraproject.org/sources/dirsrv.repo

Then, you’ll need to import a couple of keys in order to verify signatures of the packages we’ll install
later:

sudo rpm --import \
http://archives.fedoraproject.org/pub/archive/fedora/linux/core/6/i386/os/RPM-GPG-KEY-fedora
sudo rpm --import \
http://archives.fedoraproject.org/pub/archive/fedora/linux/core/6/i386/os/RPM-GPG-KEY-Fedora-Extras

Next, install some prerequisite packages (you could do this first – these come from standard
repositories, not the new ones we added):

sudo yum install svrcore mozldap perl-Mozilla-LDAP libicu

You’ll need jss, and I wasn’t able to get it via a repository, so I downloaded it using a URL directly:

sudo rpm -ivh http://download.fedoraproject.org/pub/fedora/linux/extras/6/x86_64/jss-4.2.5-1.fc6.x86_64.rpm

Next, install ldapjdk (used by the FDS console application), and finally, the directory server itself:

sudo yum install ldapjdk
sudo yum install fedora-ds

WIth these packages installed, the next thing to check is that permissions are set up correctly, otherwise the initial setup script will fail:

sudo chown -R nobody:nobody /var/lock/dirsrv; sudo chmod -R u=rwX,go=
/var/lock/dirsrv
sudo chown nobody:nobody /var/run/dirsrv; sudo chmod -R u=rwX,go= /var/run/dirsrv

Finally, run the setup script which was installed with the fedora-ds package:

sudo /usr/sbin/setup-ds-admin.pl

Populating the Direcotory

The directory initially consists of a top-level entry representing the domain, and by default, FDS creates for you two “organizational units”, which are subtrees representing “People” and “Groups”. I’ll create an LDIF file for the Groups first, but there’s no reason to go in any particular order. We’re just adding data, and LDAP isn’t relational: you can add People objects who are members of Groups that aren’t in the tree yet. Here’s my LDIF file for the groups:

dn: cn=wheel,ou=Groups,dc=example,dc=com
objectClass: posixGroup
objectClass: top
cn: wheel
gidNumber: 1000
memberUid: jonesy
memberUid: tasha
memberUid: molly 

dn: cn=eng,ou=Groups,dc=example,dc=com
objectClass: posixGroup
objectClass: top
cn: eng
gidNumber: 1001

For the moment, only ‘wheel’ contains any actual members. No biggie, you can add members to groups later, or add more groups later whenever you want. Once the clients are configured, there’s no restarting of anything to get them to pick up changes to data in the LDAP data.

It’s easy to use the OpenLDAP tools to add data to FDS, but I’m going to use the FDS-supplied tool here to insert this data:

/usr/lib64/mozldap/ldapmodify -a -D "cn=Directory Manager" -w - -h localhost -p 389
-f ~/groups.ldif -c

If you’re familiar with the OpenLDAP tools, this probably doesn’t look too scary. The OpenLDAP tools require a ‘-x’ flag to bypass SASL. Aside from that, pretty straightforward.

To populate the “People” tree in FDS, or any other LDAP product, I wrote a really cheesy awk script that I can pipe the contents of /etc/passwd or ‘ypcat passwd’ through and get good results with only minor tweaking. Redirect the output to a file called ‘people.ldif’, and then you can populate your “People” tree:

/usr/lib64/mozldap/ldapmodify -a -D "cn=Directory Manager" -w - -h localhost -p 389
-f ~/people.ldif

At any time, you can verify that your FDS installation is returning results by running a query like this:

/usr/lib64/mozldap/ldapsearch -b dc=example,dc=com objectclass=organizationalUnit

I have a few more posts to follow this one. One is one getting SSL/TLS working (either one, perhaps both), creating a root CA and setting things up with certutil, another on getting the RHEL 4 and 5 clients to use LDAP, and another separate one for configuring autofs to talk to LDAP, which is a little different between RHEL 4 and 5. Subscribe to this blog in your reader to be updated as those posts come out over the next 2 weeks.