Continuing my series of things I take for granted, over the course of my life I have used UNIX-derived operating systems: AIX, SunOS, Solaris, OpenBSD, FreeBSD, Linux, and now OS X (which is based on FreeBSD).  I first learned how to use UNIX via the command line, dialing in to systems and telneting to others.  (As an aside: at the same time, I also got to learn VAX a little.  Yay.)  Because I first used UNIX-like systems via the command line, the command line is very comfortable to me.  I know it isn’t comfortable to most people, and there are in fact very good GUI (graphical user interface) equivalents to most of what you can do via the command line.  The nice thing about the command line is that you can do some very powerful stuff quickly, without the mouse.  Here are a couple of things I take for granted on the command line.  This is probably most relevant for people using OS X, or people with cygwin installed on Windows.

1. du -cks * | sort -n
   

   This one I use so much I create an alias for it.  ‘du’ looks through every entry in the current directory that ‘*’ matches, and prints out how much space in kilobytes that entry takes.  ‘sort’ then sorts the entries numerically.  The output is a sorted list of stuff in the directory where the biggest stuff is at the bottom.  This command is useful for figuring out where all your hard drive space is going (in my case, my Aperture library).

2. vi $!
   

   This one is particularly geeky and really only useful for people who already know vi (a text editor that you hit ‘ZZ’ to quit out of).  In bash, ‘$!’ means get the last argument from the previous line.  If you do ‘ls x’ and then ‘vi $!’, bash converts this to ‘vi x’.

3. history

   Shows the commands you have typed previously.

4. man <<command>>

   Goes to the manual entry for that command.  For example, ‘man cat’ tells you how the cat command works.

5. pydoc -p 8080

   Runs a local web server at port 8080 (that you can access via http://localhost:8080) with python documentation for your installed modules.

6. cp file{,-20100131}

   Another bash trick; bash will expand whatever’s in the {}s into a list.  In this example, bash converts this into “cp file file-20100131″.  I use this to take simple backups of files before I destroy them accidentally.

7. <<command>> | sort | uniq -c | sort -n
   

   Take the outputs of command and send it to the sort command.  Unique removes duplicates from output and prints how many rows there were.  Effectively this prints out unique lines, sorted by how many times they occurred.

8. command | cut -d' ' -f 2

   ‘cut’ will use the specified delimiter (-d) and print out the specified field (in this case, the second field).  For example, if the input were “Hello there fred”, this particular command would print out “there”.

9. svn <<commands>>

   I should probably learn how to use a newer version control tool.  Subversion (svn) is a tool for version control; it lets you check out files, edit them, and check them in.  For someone like me who makes lots of mistakes, subversion and its ilk are a lifesaver because it helps me find out where things started going wrong.

10. ifconfig -a

    Shows information about network interfaces.  Helpful for finding out your IP address(es).

11. grep -rli 'something' .

    Very very helpful file-searching tool.  grep -rli searches all subdirectories for the string specified.  It is slow.

12. find . -mtime -3 -ls

    Extremely helpful file-level search tool, in this example to find all files within the directory tree modified in the last three days.

13. locate x

    Search the locate database of all files on the system for files that have a given string in them, in this case ‘x’.  Requires that you’re running a job to update the locate database from time to time.

Well, those are just a couple that come to mind.  The command line can be really helpful for working with your filesystem and for finding stuff.  Fortunately, there are now also tools such as Quicksilver and the OS built-in search tools that can do a lot of this same stuff, but if you do need to do a lot of messing around with files (especially groups of files), learning the command line can really pay off.

This article continues my series on describing what I already take for granted on my computer.  At work, running Windows XP, I use two programs to keep my passwords and files secure:

1. Password Safe, an open-source program for storing passwords that was originally written by Bruce Schneier.  I chose this program because Bruce Schneier is awesome.  (On my Mac at home I use Apple’s “Keychain Access” instead.)
2. TrueCrypt, an open-source disk encryption program for storing files securely.

Passwords

I keep Password Safe running all the time.  You can double-click Password Safe from the tool bar.  The icon is green when the safe is “locked” and red when the safe is “unlocked.”  To unlock the safe, Password Safe prompts you for a master password.  Once you type this password in the safe becomes unlocked.  When you haven’t used the safe for a couple of minutes it automatically locks again.

You can group passwords into categories, e.g. “listserv passwords.”  Whenever you double-click a password it is copied into the clipboard.  You can also tell Password Safe to go directly to a URL associated with that password.  The clipboard is purged whenever Password Safe is minimized.

With this system, your “master password” gives you quick access to all the passwords you’ve saved.  Password Safe can randomly choose a password for you, so that you don’t even need to know the individual passwords–just the master password that gives you access to your stored passwords.

Files

Once you install and run TrueCrypt, you will see a new icon in the task bar that you can double-click to bring up the TrueCrypt application window (pictured).  This window lets you “create a volume” or mount a created volume.  Basically what’s happening here is that you

1. create a secure file in which to store confidential materials, e.g. a 1 GB encrypted file on your computer somewhere.
2. you use TrueCrypt to “mount” (open) this file as a new disk drive, e.g. “H:”.

To create a volume you use the “create volume” button.  I then selected a “create an encrypted file container” as a standard truecrypt volume.  The file name should end in “.tc” for TrueCrypt so you can double-click it and have TrueCrypt open.  You set a file size–and be careful here, because to the best of my knowledge you will be forever limited to this file size.  Finally, you can set a password and/or a “key file.”  I create a hugely long password and–you guessed it–put this password in Password Safe.  That way I don’t need to know the password.  If you want, you can also create a “key file” e.g. a file on a CD or USB key that is required to open the TrueCrypt volume you’re creating.  Just be careful–if you lose the key file you essentially lose all your data.  (So at least make a backup!)

To open a volume, you just “select file…”, find the file, enter the password, and tell TrueCrypt what letter to use (e.g. “H:”).  Then TrueCrypt will mount the drive, and you can use it as you see fit.  If you choose a standard letter, you should also tell other programs to be careful so they do not index or back up your encrypted contents.  For example I use Google Desktop, so I have told Google Desktop not to index the stuff on the “H:” confidential drive.

You can tell TrueCrypt to dismount the drive after 15 minutes of inactivity, if you want, so there’s less of a chance of people seeing the data.

Why would you want this?  Well, I use a TrueCrypt partition to store all employee performance reviews and other confidential data such as budget information.  If your hard drive is stolen, the thief can easily read unencrypted files–but it is much more difficult for intruders to break into encrypted TrueCrypt volumes!

One of my goals for 2010 is to blog about things I take for granted when I use the computer.  If you’ve used MS Word styles before, this article may be old hat to you.  However, if you haven’t used MS Word styles, I hope to introduce you to a whole new way to format your documents–so you will never have to use the “Font” menu again.

MS Word 2007 made it a little easier to find styles: there’s a “Styles” menu in the ribbon under the “Home” tab.  You may have already been tempted to click in this menu.  When you click on a style, it applies to the current paragraph. So if you click on “Heading 1″ while in a paragraph, all of the sudden everything in the paragraph gets gigantic and blue.

So, there’s a styles menu.  Why would you want to use the styles menu?  For me, the easiest way to understand styles is to see all the options for a given style.  To do this, you can click on the little styles arrow in the corner to show the Styles menu.

You should now see a list of styles.  Right-click on “Heading 1″ and select “Modify…”.  This will take you to the “Modify Style” menu.

Once on the “Modify Style” menu, note a couple of things:

• “Style based on“: For computer scientists, you can think of this as the style’s “super-class”.  That is, this is the base style from which the current style is derived.  In this example, “Heading 1″ is based on “Normal”.
• “Style for following paragraph“: What should happen when you hit “Enter”?  Do you want to keep typing in this style, or do you want to switch to another style?  In this example, “Heading 1″ is assumed to be one paragraph: why would you want to keep typing in Heading 1 for several paragraphs?  Therefore the style goes back to “Normal” when you hit enter.
• The little block below the preview shows all modifications from the base style.  In this case, we see every possible change from the “Normal” style.  For my MS word installation, which includes some extra language packs, we see some references to “(Asian)” fonts but mainly we see that the font becomes 14 point, bold, there’s 24 point of space above the heading, and there are several other differences mainly having to do with magic associated with “Heading 1″ that you don’t see.

So, how could this be useful?  Well, one style I created is for “dictionary entries.”  Dictionary entries are underlined with a roman numeral, e.g.

1. Test: Definition of test

To do this, I created a new style called “Dictionary.”  Then I used the “Format” menu to say that the style should be underlined, and that the “Numbering” should be roman numerals.  While writing my document, I select the style from the “Styles” menu when I’m ready to create a dictionary entry.

Why?

Why go through this effort?  Why use styles rather than just create a list and remembering to use underline?  For me there are a few reasons:

1. You can save your styles to use over and over.  If you set up your styles in a blank document, and “Save as…” a “Word Template (.dotx)” or “Word 97-2003 Template (.dot)”, then whenever you double-click on these templates they will create a new blank document that already has these styles in it.
2. You can update your entire document formatting through changing styles.  My most hardcore use of MS word was in college, when four of us were simultaneously writing a shared 60-page document.  We used “compare & merge documents” a lot, plus we used styles.  MS word automatically creates style entries for weird stuff you put in your document; you can then select all the weird styles and convert them to normal styles.  Then, if you decide that all of your “dictionary” elements should use a different type of numbering, you can change them in one fell swoop.
3. MS Word does magic for certain types of styles.  Most notably, MS Word will automatically create a table of contents for you.  It does this by looking through your document for paragraphs with the “Title”, “Heading 1″, “Heading 2″, and “Heading 3″ styles.  It assumes that Heading 2 entries are sub-entries of Heading 1, and so on.

So, that’s my own little intro to MS word styles.  For the official story, check out Microsoft’s “Style basics in Word” article.

I’m trying to start going through my LaterLoop queue, which I’ve apparently had for over a year now. Thus, I finally saw this video, “Hug a Developer”:

I have been learning how to write Django code for Google App Engine.  (Django is a Python framework, which Google chose to incorporate into Google App Engine.  Django is cool.)

Google has a few Django customizations, which they package in “google-app-engine-django“.  For example, the base class that your Django models use is supposed to change to BaseModel, a django helper front-end to “db.Model”, a Google Bigtable database model.

Today I wanted to implement search for one of my Django models.  Google has an article about setting up search using SearchableQuery, and some discussion posts about how to use ext.search and search.searchableModel, but nothing about how to use search.searchableModel with BaseModel.

The solution (which I probably shouldn’t be happy with):

1. Go to $ROOT/appengine_django
2. Edit models.py
3. Put “from google.appengine.ext import search” towards the top
4. Copy the “class BaseModel” class
5. Paste it in below itself, and change the top line of what you pasted to “class SearchableModel( search.SearchableModel ):”
6. Have your Django models inherit from SearchableModel

This allows you to say YourModel.all().search( ‘x’ ).  w00t.

I’m trying to figure out how to make Django 0.96 work with Google App Engine.  Here are the resources I’ve found so far:

• Django’s 0.96 documentation (since the latest Django version is 1.0, and Google App Engine supports 0.96)
• Running Django on Google App Engine, which I wish I hadn’t seen until after I’d read
• Using the Google App Engine Helper for Django.

I’ve found a few issues so far, that weren’t explicitly mentioned in these notes…

• All models should use BaseModel from the django helper (this is stated)
• All Python module names are relative to the root directory (e.g. myapp.views) rather than relative to the parent of the root directory (e.g. myproject.app.views).
• Generic views do not work.  You have to re-implement the views yourself.
• Once using the app engine helper, do not use dev_appserver.py.  Using “python manage.py shell” will consult a separate content store and you will be confused.  Instead use “python manage.py runserver”, the Django-esque way of doing things.
• Restart the development server after using “python manage.py shell”.  I’m imagining there’s a caching issue.
• All sorts of basic Django model syntax is broken.  YourClass.get( pk=1 ) becomes YourClass.get_by_id( 1 ), for example.  YourObject.choice_set.create does not work either.
• The object ID is stored at object.key().id()

I’m sure there’s a bunch more to learn, but I’ve finally gotten the Django tutorial example to work.