Derek Slager

Tuesday, September 11, 2007
A Better .NET Regular Expression Tester

Because the only other online tool I could find for testing .NET regular expressions was slow and covered with ads, I decided to write a simple AJAX regular expression tester. It's certainly not fancy, but it works for me.

OptionsCultureInvariant ECMAScript ExplicitCapture IgnoreCase IgnorePatternWhitespace Multiline RightToLeft Singleline
Input
Source
Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
Pattern

96 CommentsPosted at 8:52 PM
Saturday, March 24, 2007
Emulating Vista's User Directory Structure on XP

Perhaps the best new "feature" of Windows Vista is the Unix-inspired re-organization of user home directories. That is, instead of placing user-specific files under the abhorrent Documents and Settings directory, each user's folder resides under the command-line-friendly Users directory in the root system drive. Even better, your music is no longer considered a document. It's hard to argue against the fact that C:\Users\Derek\Music is cleaner than C:\Documents and Settings\Derek\My Documents\My Music (or in some cases, the 8.3 friendly C:\DOCUME~1\Derek\MYDOCU~1\MYMUSI~1, ack!).

I use Vista on a couple of my computers, and I've grown to appreciate the new folder structure. Gone are the days of hacking together your own home directory structures just to try and cope with the unfriendly madness (who hasn't created a folder or two under C:\ just to make them typable?). Unfortunately, I still use several machines running XP, so it's hard to rely on this sane structure -- leading to grotesque hacks such as functions called ghetto-dosify in my .emacs file (I wish I were kidding).

There are some additional advantages to utilizing the built-in home directory structure rather than creating your own. For example, the Run dialog resolves path reference relative to your home directory. So, if you type Start -> Run -> ., your home directory will open. Sub-folders can be accessed just as easily -- typing Start -> Run -> dev\blog opens my blog source tree. Also, many tools (backup, indexing, etc.) will only consider content in your actual profile directory (the one that Windows knows about).

Another nice benefit is that the compact path structure is Cygwin-friendly, so you can safely set your HOME environment variable (also used by Emacs, of course). Thus, ~/Desktop in Cygwin matches ~\Desktop in Powershell. I keep my HOME directory synced across many machines (Vista, XP, and Linux) using Subversion, and it's nice being able to keep everything in the same place.

Fortunately, we can make some relatively simple tweaks to our XP systems to make the structure more closely resemble that of our Vista and Unix friends.

Junction, Junction, What's your Function?

Chances are you've been running your operating system for a while, and you've got a bunch of clutter under your Documents and Settings folder. More importantly, you probably have a bunch of applications with stored absolute paths under that folder, not to mention all of those common dialog pointers. So obviously, just renaming the Documents and Settings folder is going to cause more problems than it'll solve.

If XP had symlinks (another feature added in Vista), of course, we'd have an easy solution. We could simply type ln -s DOCUME~1 Users, and we'd be off to a good start. We could update environment variables to use our new path structure, and we wouldn't break the old path references. Fortunately, XP does support directory symlinks ... well, sort of. XP's default file system, NTFS, supports something called a "junction point". For our needs, it'll do.

While junctions are supported by NTFS, there is no built-in tool for creating them. Enter Junction, a simple command line tool for creating (and viewing targets of) NTFS junction points. With the tool located in your PATH, run the following command sequence to create the link:
```
C:\Documents and Settings\Derek>cd \
C:\>junction Users "Documents and Settings"

Junction v1.04 - Windows junction creator and reparse point viewer
Copyright (C) 2000-2005 Mark Russinovich
Systems Internals - http://www.sysinternals.com

Created: C:\Users
Targetted at: C:\Documents and Settings
```
Special Folders

At this point, we can access our home directory under the Users path, but unfortunately Windows has no idea that we've actually intended to change our home directory. For example, starting a fresh command line session stills shows that old, annoying Documents and Settings path.

To fix this, we need to dig into the registry. Under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList, there should be a number of keys in the form S-x-x-xx-.... Browse through the keys until you find the one with a ProfileImagePath value ending with your user name (e.g. %SystemDrive%\Documents and Settings\Derek). Change this value to use the new path (e.g. %SystemDrive%\Users\Derek). You will need to restart Windows for it to pick up the change.

After rebooting, you should default to the new Users path when starting a fresh console session. And because it's actually a junction point, all of your existing files should be right there with you. This is a nice start, but there are still some annoyances: My Documents should be Documents, My Music (and its similarly named friends) should also drop the silly "My" prefix, and ascend one directory (unless you think music is a document, of course).

Again, junction points are exactly what we need. Use the following command sequence to set them up:
```
C:\Users\Derek>junction Documents "My Documents"
...
C:\Users\Derek>for %f in (Music Pictures Videos) do junction %f "My Documents\My
 %f"
...
```
You might also want to hide the My Documents folder, so you don't see two references to the same folder when browsing your home directory. You could just make it hidden, but if you're like me you have hidden folder display enabled. In this case, we just need to make it hidden and give it the system attribute using the following command: attrib +S +H "My Documents".

Our paths are now much prettier, but we do get stuck with the ugly default folder icon. Fortunately, you can right click on the folder whose icon you wish to update, click Properties, and change the icon in the Customize tab (you must do this before the next step, where they are assigned as special folders). I use the normal icons in shell32.dll for Documents and the like. For my root home directory (Derek), I use user-home.ico (attached to this post) from the Tango Desktop Project. The icon was conveniently translated to ICO format by Ben Brown. You might receive a warning about "enabling task folders", this is safe to ignore.

As with the Users junction point, we need to tell Windows about our new paths. For this, we'll use a Microsoft tool called Tweak UI. Install the tool, start it (Start -> Run -> tweakui), and select the Special Folders item under My Computer in the tree control. Here you can select each path for which we created a junction, and assign the new path.

Common Dialogs

We've solved our ugly path problem, but overall usability still leaves something to be desired. For example, our common dialogs all have direct pointers to our Documents folder, but it's still rather painful to get to our root home directory to, say, open a music file.

Fortunately, Tweak UI comes in handy here as well. Under Common Dialogs, select Places Bar, and choose Custom Places Bar. Here, you can define up to five paths that you will have easy access to using the common file dialog used by most Windows applications.

Here's the new dialog in action:

Another nice convenience is having a home directory link in your start menu, as below (simply drag the link there to create the shortcut):

Attachments
- user-home.ico
22 CommentsPosted at 3:03 AM to Categories: Windows
Saturday, February 24, 2007
Screencast: Formatting a CSS File with Emacs

When I wrote The Case for Emacs, the main point I was attempting to convey was that Emacs is an amazingly effective editor even without the customizations it's so famous for. Right out of the box, you can do some pretty incredible things with its broad set of built-in commands. And if a simple Emacs configuration is good enough for Donald Knuth, it's good enough for us, right?

The other day I came across a CSS file that was in need of some formatting tweaks, so I slapped together a quick macro and fixed it up. Nothing special, but it got me thinking -- why not take this example to an extreme, creating the ugliest CSS file of all time, and create a screencast of cleaning the file up using a bare bones Emacs configuration? Well, here we are.

The CSS file you see getting the Emacs treatment in the screencast is not real. I intentionally created about the ugliest file I could, butchering indentation, casing, structure, mixing tabs and spaces ... you name it (the CSS file is attached, if you're especially curious).

I'm not necessarily suggesting that Emacs is the best tool for the specific task we're performing here, but it's a pretty broadly understood file format, so I thought it would be interesting way to demonstrate some of Emacs core functionality. If your favorite CSS editor has a magical "auto-format" button, by all means use it -- it's better than this strategy to be sure, but it's a lot less flexible!

The Emacs instance you see in the screencast is extremely close to a stock distribution. I made the following modifications, in the interest of making it easier to see what's going on:
1. Disabled the tool bar and menu bar.
2. Loaded mwe-log-commands.el, for demonstrating the keys pressed.
3. Enabled the downcase-region command, normally disabled.
4. Loaded a simple CSS mode for font locking.
5. Changed the default font to a narrower version, Consolas.
On with the show.

Here's a breakdown of how we attacked the file.
1. Converted tabs to spaces with untabify. This is a good first step when encountering a file as hideous as this one.
2. Normalized all spacing by compressing multiple spaces and newlines to a single space, using a regular expression replacement.
3. Added newlines after all open braces ({) using a macro. Also used just-one-space before the brace to make the spacing consistent.
4. Added newlines after all semi-colons, again using a macro. Also compressed space in front of the semi-colons, and added a space after the colon delimiting the property from the value.
5. Used another macro for adding newlines before and after the close braces. I added the extra spaces before using delete-blank-lines because the spacing varied based on whether or not a trailing semi-colon was present.
6. Killed a few empty blocks, using backward-paragraph to quickly navigate blocks.
7. Executed another macro to make the property name case consistent (made them all lower-case).
8. Sorted properties by name within each block using a macro to regionize the block and call sort-lines.
9. Compressed expanded forms of margin and padding specification to a single line, using a multi-line regular expression replacement with a reference to a captured group in the replacement string.
10. Performed a few simple manual cleanups, and updated the messaging.
Attachments
- fugly.css
20 CommentsPosted at 11:14 PM to Categories: Emacs
Friday, January 26, 2007
Emacs Hack #3: Compile Emacs from CVS on Windows

In previous hacks, we learned how to install and configure stable binary builds of Emacs. While the stable version (currently 21) is the best version to run for most users, you may be brave or curious enough to try one of the newest pre-release versions, 22 or 23. Because there are no official binary builds of Emacs beyond version 21, you will either need to install it from an unofficial source, or compile it yourself. This hack covers the latter.

Get the Source

The first step in compiling your own version of Emacs is getting the source. It is sometimes possible to obtain gzipped archives of the Emacs source at a given point, but it's typically more convenient to grab the source from CVS (this also ensures that you're using the most up-to-date version of the given branch).

If you don't already have CVS installed, fetch a recent version of cvs.exe from its distribution site, and place it somewhere in your path.

Next up, we pay a visit to Savannah, the GNU development site. The Emacs project page contains details on how to obtain the sources. At the time of this writing, the following command will download the HEAD version of the code (Emacs 22).
```
cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/emacs co emacs
```
If you wish to build a version of Emacs other than 22, you will need to pass a tag name to your checkout command (or a subsequent update command). For example, adding -r EMACS_21_3 would check out the code for the Emacs 21.3 release (the stable release at the time of this writing). For a sneak preview of (the very unstable) Emacs 23, use -r emacs-unicode-2.

After checking out the sources, cd to the nt directory. As a safety measure, issue a cvs up -kb command here to make sure that all files in the directory have proper line endings. At this point, the source tree is ready to be built.

Prerequisites

The Emacs build process requires a handful of tools that probably don't already exist on your system. These include GNU versions of tools with Windows equivalents (cp, rm), as well as tools that are often unique to a GNU system (makeinfo). If you want your build to support images, you will also need a variety of libraries for rendering image formats. All of these tools are available from GnuWin32, a project which provides native Windows binaries of GNU tools.

The packages you will need to install are listed below. In each case, I suggest installing the latest "setup" package, which will run an installer and place the binaries in a consistent location.

CoreUtils

This package contains a variety of tools, and is required in order to build Emacs. Specifically, it contains cp and rm.

TexInfo

This package contains makeinfo, which generates Info documentation from the texinfo sources in CVS. While technically optional, I strongly recommend installing it.

If you wish to build Emacs with image support (optional), you will also need the following image libraries (from the GnuWin32 Packages page:
- LibPng
- Jpeg
- GifLib
- Tiff
- Xpm
- Zlib
At the time of this writing, the Xpm library is missing a required header file (simx.h). You can either get it from the source package, or download it from the attachments in this post. This file should be placed in the GnuWin32 include directory.

Compiling with MinGW (GCC)

The simplest way to build Emacs is using MinGW, a collection of freely available tools for building native Windows binaries. The package includes the GNU C compiler, a port of make, and various header files (including those from the Win32 API, such as windows.h).

The first step, of course, is to obtain the MinGW distribution if you don't already have it installed. The current version at the time of this writing is 5.1.3, and is downloadable at SourceForge.net (via the project page). Install to the location of your choosing, being sure to select the following components:
- MinGW base tools
- MinGW Make
After installing MinGW, add its bin directory to your path using a normal Windows Command Prompt session (set PATH=%PATH%;C:\MinGW\bin). Next, run configure.bat from the nt directory as follows to build a Makefile:
```
configure.bat --no-debug --with-gcc
```
If you installed the libraries for image support, you will also need to pass the appropriate include path to the configuration script. If you installed to a directory with a space (the default), use the DOS name of the directory as demonstrated below:
```
configure.bat --cflags -IC:\Progra~1\GnuWin32\include --no-debug --with-gcc
```
If all goes well, you'll get a message telling you to run gmake to build Emacs. We're not quite ready for that yet, however. Because we got our source from CVS, we need to perform a "bootstrap" build. This creates a bootstrap Emacs binary to build autoloads and byte compile the Elisp (.el) files in the distribution. Begin the build process as below (mingw32-make is the MinGW name for gmake):
```
mingw32-make bootstrap
```
After the bootstrap completes, you're ready to build the source code. You'll also want to build the info files (make sure makeinfo.exe is in your path) before installing. Run the following commands in sequence to finish compiling and install your Emacs build.
```
mingw32-make info
mingw32-make
mingw32-make install
```
By default, Emacs will be installed "in place". If you prefer to install it somewhere else, run configure.bat with a --prefix <dir> argument pointing to your preferred installation directory.

Compiling with MSVC

The most popular C/C++ compiler for Windows is MSVC, Microsoft's Visual C++ compiler. Traditionally, this compiler was only available commercially, when purchased as part of Microsoft's Visual Studio development product. In recent years, however, Microsoft has made simple versions of the compiler available free of charge. Fortunately, the simple versions suffice for building Emacs.

At the time of this writing, the VC8 compiler (part of Visual Studio 2005) is the most recent version available. Unfortunately, Emacs cannot currently built with this version of the compiler without modifying the source. The only other freely available version of the MSVC compiler was made available as part of the Visual C++ Toolkit 2003. Microsoft no longer distributes this compiler (it was replaced by Visual C++ 2005 Express Edition), but if you already have it installed, or happen to have downloaded it previously (the file name is VCToolkitSetup.exe), you can proceed to build Emacs using this compiler. If you have a commercial version of Visual Studio 2003 installed, the steps should be very similar.

The Visual C++ Toolkit installation is very basic. In order to build Emacs, you need a variety of header files in addition to those packaged with the toolkit. To obtain these, you will need to download and install a (pre-Vista) version of Microsoft's Platform SDK. Because you only need a base set of components, the Web Install method is probably optimal. The following screen shot shows the components you need to have selected:

Next, add cl.exe to your PATH in a normal Windows Command Prompt window. The easiest way to do this is to run the vcvars32.bat script packaged with the download, e.g.:
```
"%ProgramFiles%\Microsoft Visual C++ Toolkit 2003\vcvars32.bat"
```
Next, you need to add the Platform SDK paths to your environment. For example:
```
set SDKROOT=%ProgramFiles%\Microsoft Platform SDK for Windows Server 2003 R2
set INCLUDE=%INCLUDE%;%SDKROOT%\Include
set LIB=%LIB%;%SDKROOT%\Lib
```
If you are building with image support, you'll also need to add the GnuWin32 paths:
```
set INCLUDE=%INCLUDE%;%ProgramFiles%\GnuWin32\include
set LIB=%LIB%;%ProgramFiles%\GnuWin32\lib
```
Normally we'd be done at this point, but we're still missing a couple of pieces. The first is setargv.obj. Fortunately it's only missing in binary form -- it can be generated from the Platform SDK's CRT source directory. To build the required file and place it in the Platform SDK's lib directory, run the following command:
```
cl /c /D_CRTBLD /I"%SDKROOT%\src\crt" /Fo"%SDKROOT%\lib\setargv.obj" \
    "%SDKROOT%\src\crt\setargv.c
```
We're also missing a few essential binaries the build process expects: nmake.exe, rc.exe, and lib.exe. All of these files are packaged with commercial versions of Visual Studio, but you'll need to find them elsewhere if you're using the Toolkit. The first two can be obtained free of charge by installing the .NET Framework SDK. Make sure you add its bin directory to your path after installing. lib.exe is simply an alias for link /lib, so a simple batch file (lib.bat) will suffice. A sample is attached to this post -- place it somewhere in your PATH.

Finally, we're ready to build Emacs using MSVC. As with the MinGW build, the process begins with running configure.bat:
```
configure.bat --no-debug --with-msvc
```
Things are pretty straighforward from here. As with the MinGW build steps, we need to start with a bootstrap build, after which we can build the complete distribution. Refer to the MinGW steps for details:
```
nmake bootstrap
nmake info
nmake
nmake install
```
Optimized Builds

One of the advantages of compiling from source yourself is that you can optimize builds for your platform. For example, if you're using a modern AMD or Intel processor with SSE extensions, you can enable the compiler to generate optimized code for your processor. To build an optimized version of Emacs, you will need to pass arguments to the compiler by way of configure.bat's --cflags argument. To build a version of Emacs optimized for an SSE2-capable Athlon or Pentium 4 processor with MSVC, for example, you would run the script as follows:
```
configure.bat --no-debug --with-msvc --cflags /O2 --cflags /G7 --cflags /arch:SSE2
```
For a similarly optimized build with MinGW (GCC), use:
```
configure.bat --no-debug --with-gcc --cflags -msse2 --cflags -O3
```
Optimized builds will typically be incompatible with older processors that do not support the selected extensions (in most cases, they will crash at runtime). If you are creating an Emacs build to share, minimize optimizations that require specific processor features such as SSE.

Final Steps

If you built Emacs with image support, you need to copy the runtime DLLs to Emacs' bin path (or your system path, if compiling only for a single system). You can copy them manually, or script it as below (specific file names may vary slightly if versions change):
```
for %f in (giflib4 jpeg62 libpng13 libtiff3 xpm4 zlib1) do \
    echo copy "%ProgramFiles%\GnuWin32\bin\%f.dll" ..\bin
```
Attachments
- simx.h
- lib.bat
17 CommentsPosted at 4:28 AM to Categories: Emacs
Friday, January 05, 2007
Emacs Hack #2: Manage Emacs Instances with gnuserv

If you followed Hack 1, you can now launch Emacs and open files from within the interface. On Windows (and in some other environments), you can also drag files to Emacs from your window manager.

But what if you want to send a file to Emacs from a console session, or via a shortcut? You could invoke Emacs using runemacs (or create a shortcut to it), but every time you do that you end up with a whole extra instance of Emacs -- a unique operating system process which is independent from any previous instance(s).

In some cases, this might be what you want. In the rare case of an Emacs crash, for example, you would not lose any active buffers in the other instances (note that Emacs' auto-save facility would probably mitigate the damage here). Most of the time, however, you'd rather open the file in an existing instance of Emacs. This has a number of advantages:
- Memory is conserved by sharing a single process.
- Shortened start-up time since Emacs is already loaded.
- All open buffers in the shared instance can be quickly accessed from any Emacs frame.
- The buffer list can be used to see every open file across frames, and to perform actions on them.
- Dynamic abbreviations (covered in a future hack) can be sourced from a larger set of files.
- Contention issues (multiple processes accessing the same file) are avoided.
Fortunately, there is a small client / server program called gnuserv which enables us to do just this.

Installing

Installing gnuserv consists of two parts -- some platform-specific binaries (gnuserv, gnudoit, gnuclient and, on Windows, gnuclientw), and an Emacs Lisp file which we'll load into Emacs. The Lisp code will spawn the gnuserv process after Emacs has started, listening for received commands on gnuserv's standard output stream.

On Windows, the first step is to unpack the ZIP file containing the gnuserv binaries (on Unix and derivatives, simply install the gnuserv package for your distribution). The Windows port is available at Guy Gascoigne - Piggford's site, or attached to this post if the site is unavailable. Extract the binaries to a directory in your %PATH% (see Hack 1 for some tips on setting up your path), e.g. C:\Program Files\gnuserv.

Next, we need to install the Emacs Lisp portion of gnuserv. This involves placing the gnuserv.el file (attached to this post) somewhere in Emacs' load path. The easiest way to do this is to copy the file to the site-lisp directory, typically located under your base Emacs installation directory (Windows) or at /usr/local/share/emacs/site-lisp (Unix and its derivatives). Files placed here are automatically available to Emacs.

Configuring

We've installed all of the pieces necessary for gnuserv, but nothing has really changed in terms of Emacs' behavior. Even though we've added gnuserv.el to its load path, we still need to instruct Emacs to load the file. We'll do this by adding some initialization code to our .emacs file (see Hack 1).

From within Emacs, type C-x C-f (that's the control key plus x, followed by the control key plus f -- you can keep the control key pressed the whole time). This invokes Emacs' find-file function, which prompts you for a file path to open using the minibuffer (the bottom line in the frame). Emacs will present you with a suggested path. Type ~/.emacs, press Tab (Emacs will remove the suggested path as part of its completion algorithm), and then press Enter to open the file.

If you're a Windows user, this path might look strange. The tilde (~) character is simply a handy shortcut for the folder represented by your %HOME% environment variable, and the forward slash is Emacs' preferred path separator (that is, C:/Users is preferred to C:\Users). The path syntax for the home directory is not only convenient to type, it also makes your Emacs configuration more portable. For example, ~/Desktop always maps to your Windows desktop, regardless of your login name or other machine-specific details.

At this point, you should be staring at a blank buffer. Add the following lines to your .emacs file:
```
(require 'gnuserv)
(gnuserv-start)
(setq gnuserv-frame (selected-frame))
```
This might look somewhat strange -- it's actually just some simple Elisp which tells Emacs to load and start gnuserv. The first line calls the built-in require function, which tells Emacs to load gnuserv if it hasn't already done so (this causes Emacs to go find gnuserv.el in our load path). The next line calls a function in gnuserv.el, which starts (or restarts) the gnuserv process. The last line is an optional customization which tells Emacs that it should open new files from the clients of gnuserv in its startup frame (if you find that you prefer to have multiple Emacs frames, comment this line out by prefixing it with a ; character, or delete it entirely).

Emacs reads its .emacs file and evaluates these lines each time it starts up. You can either restart Emacs now, or press M-x (M stands for meta, which is typically Alt), type eval-buffer, and press Enter (this tells Emacs to evaluate the contents of the current buffer without requiring a restart).

Launching Emacs

At this point, you should have an Emacs instance running with gnuserv started. So far, so good. Now it's time to use the client programs which came with the gnuserv distribution.

gnuclient and gnuclientw

The gnuclient program can be used to open a file in a running instance of Emacs. For example, typing gnuclient README from a console session will open a new buffer for the file README (even if it doesn't yet exist). If Emacs is not currently running, it will automatically be started (at which point it will evaluate its .emacs file, loading gnuserv).

You may notice that when a file is opened using gnuclient, a message is displayed in Emacs' minibuffer: "When done with a buffer, type C-x #.". You may also have noticed that when you invoke gnuclient, it does not immediately exit (that is, you can't continue using the console). What's happening is that the gnuclient process is waiting for you to finish working on the file -- when you press C-x # from within Emacs, the gnuclient process exits. This is convenient in many cases -- for example, when editing a Subversion commit message. If the process returned immediately, Subversion would not be able to read the message you typed in to Emacs because there is no longer any connection between the editor and the launcher.

In other cases, however, you just want to open a file in Emacs and continue using your console session immediately. In this case, you can use gnuclientw (Windows only -- on other platforms, use gnuclient -q). This is a variation of gnuclient which returns immediately. It's also compiled as a Windows application, so it's ideal for creating Windows shortcuts (gnuclient opens an unneeded console window in these cases). I typically add a shortcut to gnuclientw in %USERPROFILE%\SendTo and %USERPROFILE%\Desktop. In practice, I nearly always use gnuclientw (with or without arguments) to start Emacs on Windows (because it's in your path, it can also be executed from the Run dialog). If you find yourself running gnuclientw a lot when Emacs is already open, try adding a -x option to "top" the Emacs frame (make it visible if it's hidden).

gnudoit

The last (and in this case, the least) client program is gnudoit, which can be used to evaluate an Elisp form. For example, gnudoit (list-buffers) will open a window within your Emacs frame listing the currently open buffers. This command is deprecated because the same behavior can be achieved using gnuclient -- on Windows, use gnuclient -e (list-buffers). On other platforms, type gnuclient -batch -eval (list-buffers). In practice, you probably won't be running these forms very often.

Attachments
- gnuserv.el
- gnuserv.zip
12 CommentsPosted at 6:34 AM to Categories: Emacs