- Wednesday, May 14, 2008
A DVCS Story
Seattle, 1990
Bob leaned back, stretched his arms and took in the view from his window. It was a lazy Sunday, and he'd just finished reading his messages on his favorite BBS. An aging Amiga decorated with pink stickers courtesy of Bob's daughter sat at his feet. His sleek new home phone was perched prominently on the corner of the desk, the cord snaking its way to the wall by way of a splitter shared with his modem.
As Bob surveyed the scene, his phone started to ring. Pausing a moment, he smiled, enjoying the harmonic ring of the new phone that he lovingly customized earlier that morning before answering.
"Hey Bob, it's Alice!" Alice, a co-worker, was one of Bob's closest friends. They had both graduated from the computer science program at the same school, managing to stay in touch over the years, usually by trading "war stories". Recently, Alice had joined Bob's group from another company. To the chagrin of Bob and his co-workers, she was brought in directly as a lead programmer. "Bah, it's just a title", Bob was fond of saying. He hid his resentment well when interacting with Alice socially.
"Bob, my life just changed forever." Bob's eyebrow slowly began to raise. His mind raced through the possibilities. Had she been promoted again? Was she moving on to another opportunity? I wonder if I'll make lead if she does!. On the verge of Bob's moment of imagination turning into an uncomfortable pause, Bob snapped out of it.
"Alice, my goodness, I've never heard you so excited."
"Okay, you know I'm not normally ahead of the technology curve," Alice continued. "But today, I just got a cellular telephone!".
Bob's eyes narrowed. A cellular phone? He'd recently seen a story about them on the evening news, and thought they were the most ridiculous thing he'd ever seen. Who on earth needs to make phone calls in the middle of the street?, Bob had thought to himself when he saw the story. Bob responded forcefully. "A cellular phone? Those crazy looking things with the huge antennas? Alice, you've got to be kidding me." Bob admiringly moved his hand across the smooth plastic shell of his phone's handset.
"I'm serious! I know they're a bit different from what you're used to, but to be honest I kind of like the styling. I didn't realize how much fun it could be to be on the cutting edge of technology. You should have seen the looks I got in the park down the street this morning!"
Bob rolled his eyes and interrupted. "Look, I'm sure it's fun to be mistaken for an FBI agent, but you can't possibly think this is rational. When the heck would you need to make a phone call in the middle of the street? I have two phones at home, and another at work. What else could I possible need?"
"I can see where you're coming from. I'm sure that to you it doesn't sound that different from your home phone. I mean, you pick up the phone, dial the numbers, and it connects to the other side. Nothing groundbreaking there. But it's so freeing to be able to talk anywhere. I can completely understand how this might seem like a novelty. To be honest, listening to myself explain it to you, it doesn't really sound that compelling. But imagine the possibilities! I'm always connected. I can have meetings in my car, and I can leave the office for a long lunch on a sunny day without worrying about missing out on anything important."
Bob chuckled, making sure he drew it out long enough to suggest that he'd heard enough. As the final note of his chuckle diminished, Bob tried to finish the conversation.
"Okay, Alice, I guess I can sort of see how that might be nice. But learning a whole new speed dial system, and working with a phone I don't understand just doesn't seem worth it for those few situations where it might be useful. I'm glad your happy with your purchase, in any case."
Alice shook her head slowly, and put in a last word before hanging up. "You'll see, Bob, just wait."
Seattle, Present Day
Bob was crouched beneath his desk, reaching for the mini-USB cable attached to the back of his computer. He plugged the other end into his battery-drained Blackberry, and its bright welcome screen came to life. Come on, come on he urged. After a brief delay, the phone's LCD dimmed and the network connected. A high pitched bell ring alerted him to a new message, and Bob quickly pressed the "Read" button. She remembered!, Bob thought to himself excitedly. His daughter had sent the text message announcing her SAT results, just as promised. I knew it!, thought Bob, proudly inspecting the score. She's smarter than her old man after all.
As Bob turned to his computer, he heard the familiar sound of his favorite classical melody. He had only last week taken an MP3 of the song and made a custom ring tone, something that impressed his daughter far more than his last promotion. Who's calling me during lunch?, he thought. Leaning back, Bob peered under the lip of his desk, eyeing the blazing white LCD screen of his phone, now perched atop his computer chassis. Recognizing the name on caller ID, he immediately grabbed the phone and answered.
It had been several months since Bob last spoke to Alice. The time between their phone conversations had grown progressively longer of late, and Bob was happy to hear her voice. It had been five years since Alice left for Silicon Valley to run the software group at a small search startup. Having moved his way up to middle management, he was always excited to hear from someone who still had their finger on the pulse of technology.
Alice got right to the point. "Bob, I am about to change your life." Bob sensed excitement in her voice. In the two decades they had known each other, Alice had embraced the role of technology evangelist, and Bob that of technology skeptic. They each enjoyed their roles, and Bob could tell immediately that Alice couldn't wait to tell him about the next big thing. He gathered up as much feigned skepticism as he could in a feeble attempt to mask his genuine curiousity, and offered a response.
"Alright, what is it this time?"
"DVCS. Distributed Version Control Systems", Alice responded. "We just migrated our entire source control system to Mercurial. I think in the first week using it we've already gained 100 hours of productivity. The developers love it. I've made it my mission to tell everyone I know."
Bob pursed his lips. She's got to be kidding, he thought, this is what she's so excited about?
Bob announced his skepticism. "Alice, Alice, Alice. I've been around a long time, and it's not quite so easy these days to put one past me. What are you really so excited about?". Alice laughed long and hard.
"Oh you old curmudgeon, here we go again. Look, we've gone through this before, and I am going to convince you that distributed version control is serious stuff. Why are you so skeptical already? What have you heard about it?"
"To be honest", he started, "I first started hearing about distributed version control when we hired on a new developer who'd been working on an open source project. He was almost as excited as you about it. He had some trouble explaining what was so great about it, so he sent the entire team a link to a video where the presenter bashed our current version control system for an hour. The guy who manages our VCS server was really offended. So as far as I can tell, distributed version control is only relevant in open source projects run by opinionated bullies who see diff & patch as a perfectly acceptable source control system. I'm an old corporate soul -- I haven't used patch since college, Alice, and I don't miss it."
"Ah, I know the video you're talking about," Alice said. "Forget about that. I'm going to make the DVCS case for you, right here, right now. Pretend you've never heard of it."
"You've got your work cut out for you. To be honest with you, nobody has ever really bothered to tell me what problem this thing solves for me."
"That's always the first question!", Alice said. "I've actually practiced this speech on a few other ex-colleagues, and everybody asks that question. Unfortunately, that's the wrong question to be asking. Instead of asking what problem it solves, you should be asking what new possibilities it offers. That's been the real win for us."
Bob leaned as far back as his chair would allow, and propped his feet up on his desk. He had been through this many times before, and he knew he was in for a long ride. He gave Alice her opening: "Alright, I'll admit to being slightly intrigued. But we're really happy with our current VCS, everybody here knows how to use it, and it handles everything we need without any issues."
Alice smiled widely. She normally had to work much harder than this to get Bob on the hook. She knew that beneath the curmudgeon, he still had a passion for technology. It was her job to make sure that he didn't lose that, and she embraced it proudly.
"Okay, I'm going to ask you a very important question, and I expect an honest answer. Has your source control server ever had any downtime?"
Bob thought for a moment. "I suppose, sure, but nothing more than the usual. I mean, our server is up all the time, really. We plan all our maintenance for the weekends, so other than the occassional hiccup ..."
Before he could finish his sentence, Alice interrupted. "Yes! The hiccups! Those 15 minutes of downtime because of the urgent security patch, the 10 minutes of slowness when two machines are pulling down copies of the repository! They happen, and you shrug them off because it's just the way it is. But do you know how much productivity you lose when somebody loses their train of thought because the server isn't available for their 'annotate' command?"
"Look, I get what you're saying. But really, these hiccups aren't very common", Bob retorted. "I mean, even if they were, you have to compare them against the cost of having everybody switch over to a whole new system. The last time we did that was a total nightmare. One of our developers even quit over it! And that didn't involve explaining to everyone this crazy new distributed source control model. I'd have a revolution on my hands."
Alice responded reassuringly. "I completely understand. I had my own reservations until very recently. One of our developers had been trying to get us to adopt a distributed source control system for a few months. She liked DVCS so much that she had found some way to use it on her local system and still interoperate with our centralized system. I was skeptical for the same reasons as you, though, Bob. I convinced myself that the server was more reliable than it was, and I tried to forget that our VPN would sometimes be down all weekend, forcing developers to come into work to make fixes. Like you, I didn't really think it was worth it."
Bob interjected impatiently. "OK, so what changed your mind?"
"Last week our system administrator told us they were going to be rebooting the server during lunch time. This wasn't a big deal at the time. Everybody had advance warning, and they made all of their checkins before lunch, just in case something went wrong. When we returned, we found our source control system in pieces, its parts splayed all over the desk. The admin had installed a few security patches, and the server wasn't booting. He said he thought it might be bad memory, but he wasn't really sure."
Alice's horror story had reminded Bob of a similar situation a year ago when something remarkably similar happened at his own company. They expected the server to be repaired in 30 minutes, but it dragged to 60, 90, then 120 minutes. By 3:30 PM most of the office had cleared out, leaving a few frustrated developers emailing files to each other as they raced towards the important deadline which was now in jeopardy.
"Bob, are you still there?" Bob was growing increasingly nervous as he enumerated in his mind the many things that can go wrong with a server.
"Yes, yes, sorry, you just reminded me of something. Please, go on."
"Okay, so the admin couldn't give us an ETA on the repair. We were feeling really helpless, and called a team meeting to decide on a protocol for getting work done while the server was down. Fortunately, one of our team members had a plan. You remember that developer I told you about earlier who was using the DVCS on her own system?" Alice was talking excitedly now, and didn't bother to wait for Bob's answer. "So, she immediately took control of the meeting, and laid out the plan. She had created a complete copy of the source system just before it went down, you know, just in case of emergency, and it was checked into the DVCS repository on her system. She gave the group some brief instruction on how to copy the repository, and in less than thirty minutes our entire team was back to work with their own copies of the source."
"Wait a second," Bob interrupted. "You're telling me that you temporarily switched your entire source control system over in a half hour in the middle of an emergency? Forgive me for being skeptical, but I'm not buying it. For starters, what happens to your revision history?"
"When she took a backup of the centralized system, she had done it via a script that preserved the entire revision history. She said that migrating from centralized systems was common, and writing the script was a breeze since most of the 'heavy lifting' was built into Mercurial."
Bob made no attempt to mask his skepticism. "Okay, I get that, and it's all very impressive, really. But what about all the work you did while the server was down? How did you get it back to the main repository once it was revived?"
"That's the best part. We didn't." Alice heard a squeak on the other end of the phone, and knowing that Bob was about to ask her if she was crazy, continued without hesitation. "The admin kept working on the server throughout the day. During this time, after everybody had their servers up ..."
Bob cut her off. "Wait, what? Servers? I thought they just checked out the source code from their co-worker's machine? Where did all of these servers come from?"
"That's exactly the difference between DVCS and a centralized system. You don't check out the source code from the server, you clone the repository from the server to your local machine. Once you've made the copy, you're a server too."
Bob was finding this DVCS concept more and more ridiculous as Alice went on. "Wait, wait, wait. Last I checked you had a team of 20 developers."
"We're up to 25 now", Alice corrected.
"And you're telling me that in response to a temporary server outage, you created 25 separate source control servers, and that's somehow a good thing? How on earth does anybody know what state the source code is in? Please tell me I'm missing something. I'm starting to believe you've gone mad."
"I know, it sounds like chaos. And frankly, it could be if we let it. That's where process comes in. In this case, everybody initially cloned the repository from the same server. So, we designated that as the official server where everybody shared their changes during the downtime."
"Aha!" Bob was sure that by the end of this conversation, he'd would bring Alice back down to earth. "So you're taking this confusing and complicated distributed system, and making it act just like a centralized system! What happens when that server goes down? You're no better off!"
Alice let out a slightly annoyed chuckle. "Not exactly. If the server we've designated as our central server goes down, which it surely will at some point, so what? That's yet another wonderful feature of a distributed system. We've got backups of the code all over the place, without even trying. Every developer's server contains a backup copy of most or all of the code from the shared repository. So, if the main server goes down, we can designate another server as the central server temporarily, or we can not worry about it, since developers can make checkins to their own servers, only sharing changes when needed."
"Wait a second!", Bob interrupted. "If everybody is making checkins to separate servers, how do you get those changes back together in the so-called central server once it's back up?"
"That, my skeptical friend, is called a merge."
"Oh, great," Bob murmered, "you mean like merging branches?"
"That's exactly what I mean. In fact, repositories and branches are pretty much the same thing in Mercurial. Whether or not there is a central server, everybody commits changes to their own servers. Period. That's the only place you can commit to. When you want to combine your changesets with those from your repository, you need to merge them."
Bob groaned. "Yuck! Branches are such a huge pain. And you're seriously suggesting that making people merge branches every single time they want to share changes is a good thing? I'm serious, Alice, if you need some help, I know some really good people."
"You actually have a point there. We used to think branches were a pain too. In fact, we only kept a single branch aside from our main repository because merging branches was such a pain. This is another huge difference between distributed and centralized systems. Because you merge branches all the time, distributed version control systems make it incredibly easy. Easy to create branches, easy to share them. Flexibility, Bob, that's what it's all about."
"I don't see what's so flexible about chaos. All this complexity, just to compensate for a few minor blips of downtime? I'm not buying it."
"Yes, initially it was just a response to the downtime." Alice sensed that she was losing Bob, a phenomenon she was quite familiar with. She softened her tone. "On the day we started using Mercurial, it was going to be a temporary thing. My thought was the same as yours, that it was handy as a temporary crutch, but too complex to keep around. However, our temporary outage ended up being not-so-temporary."
"What a nightmare." Bob said. As Alice spoke, he had been working hard trying to convince himself that this was an incredibly uncommon scenario. Geez, I guess I'd better confirm that we're backing our server up daily, Bob thought. And I wonder if I can get budget for a backup source control server?
Bob had neglected to mention that he had only a year ago spent $25,000 on licenses for his company's source control software. While he was proud to now be using the same source control system as big companies like Google and Microsoft, it left him with very little budget for hardware. As a result, their source server was running on an underpowered machine, and it was not uncommon to hear complaints about its sluggishness. Bob had also recently received the bill to renew his yearly contract for support and upgrades. That was going to take another $5,000 out of his budget, at the cost of some much needed upgrades to his developers' machines.
Alice continued. "It turned out that our source server's RAID card had died, and we were going to be down for three more days before the part could be delivered. It was in these three days that we discovered the great value of DVCS, not before. One group of our developers, for example, had previously been trying to do 'buddy builds', where they share their changes with each other before committing to the main repository. They had initially tried to coordinate this work on a separate branch, but the pain of merging so frequently was killing them, not to mention the fact that they often forgot to make their changes on the branch. Then they started emailing each other patches, but this, too was prohibitively cumbersome. To get around this, they ended up sharing their source directories to each other directly, and emailing the names of files that needed to be copied over. Changes got lost all the time, it was a total mess."
Alice was talking quickly now, hoping to stop Bob from interrupting. "The afternoon we started using Mercurial, however, all this stopped. Each developer would finish their work, commit the relevant changes to their own servers, and then make the changesets available for their colleagues to pull and merge into their repositories."
Alice had said the magic word. "Ack! More merging!"
"Yes! More merging! I'm making it sound too complicated though. You only have to actually merge files if the changes you're pulling overlap with your own changes. When you do inevitably have to merge, though, you do so after you've checked in your own changes. Thus, even if the merge goes awry, you never lose your work because of it, something that is extremely important to us. Regardless, merging is so simple and fast that it doesn't even matter."
"Well, my source control system supports branches," Bob replied, "why couldn't we do this?"
"Many of the new and interesting workflows enabled by DVCS are possible in centralized systems, but are simply too much of a pain in practice to have any chance of adoption. Plus, they can grow organically. You don't need to explicitly decide to start working on a branch for buddy builds, you can just make some checkins and choose where to send them. You're always working on a branch, in effect, since your local repository is a branch. Not to mention the fact that you don't need to ask a server administrator for permission to create a branch."
"Okay, so you can do buddy builds", protested Bob, "and if the server hiccups it's not that big of a deal. But so what? Those things aren't very useful for us. People around here just come in and do their work. Even if these benefits are as profound as you suggest, I really don't think it's worth training my entire team on some complicated new source control system."
Alice hadn't expected to sway Bob in only a single conversation, but she was nonetheless growing a bit frustrated with her progress. She had used the same sales pitch on others to great effect, but there was something missing. It just wasn't having the same effect on Bob. She continued with her defense. "The things I've mentioned so far are 'big deals'. Having a server down even for only a minute or two is a huge deal, in fact. Developer time is expensive, and knocking a developer out of the zone because of a server hiccup is, in my mind, totally and completely unacceptable. Heck, even a bit of server latency can ruin a developer's ability to stay in the zone."
"One thing I think you're forgetting is that we're using the best centralized system money can buy," Bob said. "I'm sure our server is significantly more speedy and reliable than your free, open source centralized server was. Everybody here is really happy with it. I've even heard them bragging about it to friends at other companies."
"Okay," said Alice. "I'll play by your rules, even though it's unfair. Let's assume that your server is up 100% of the time, never any downtime, never a hiccup, never a bit of slowness. Oh, and everybody has constant access to it. DVCS still offers advantages beyond simply not needing live server access to get work done."
"Oh? Like what?", queried Bob, now feeling proud of his progress towards the goal of dragging Alice back into reality.
"Like no more 'check-in races', for starters. It's a common complaint every place I've worked. The first person to commit their changes to the central server avoids having to be the one who merges changes. Thus, the next committer is forced to perform the merge, whethero r not they are the best person to merge the changes. I've actually seen people switch workstations for ten minutes so the changes could be merged by the proper resource. How ridiculous is that? With DVCS, because changes are committed locally, nobody is ever denied the ability to do so because of someone else's changes. And once they are committed, anyone can pull the changes together and merge them. If you had two developers racing towards a deadline, another less busy developer could volunteer to do all of the change merging."
"Yeah, well," Bob smirked, "I guess our needs aren't quite as sophisticated as yours. Don't get me wrong, some of the stuff you're talking about sounds interesting, and hey, if we weren't using the best centralized system that money can buy, I might be more interested. Really, whatever flaws there are in our current system, we've made the necessary adjustments. People here are happy. We're doing great work, the team gets along well. I just don't see the case for DVCS."
Alice realized that there was no more she could do. She bit her lip, resisting the urge to respond defensively. Planting the seed, she realized, was the best she could hope for with Bob. She leaned back in her chair, shook her head softly, and ended the conversation. "You'll see, Bob, just wait."
- Wednesday, October 03, 2007
BCrypt.net - Strong Password Hashing for .NET and Mono
Using raw hash functions to authenticate passwords is as naive as using unsalted hash functions. Don’t.
Thomas PtacekBCrypt.net is an implementation of OpenBSD's Blowfish-based password hashing code, described in "A Future-Adaptable Password Scheme" by Niels Provos and David Mazières. It is a direct port of jBCrypt by Damien Miller, and is thus released under the same BSD-style license. The code is fully managed and should work with any little-endian CLI implementation -- it has been tested with Microsoft .NET and Mono.
Why BCrypt?
Most popular password storage schemes are based on fast hashing algorithms such as MD5 and SHA-1. BCrypt is a computationally expensive adaptive hashing scheme which utilizes the Blowfish block cipher. It is ideally suited for password storage, as its slow initialization time severely limits the effectiveness of brute force password cracking attempts. How much overhead it adds is configurable (that's the adaptive part), so the computational resources required to test a password candidate can grow along with advancements in hardware capabilities.
Usage
Using BCrypt in your code is very simple:
// Pass a logRounds parameter to GenerateSalt to explicitly specify the // amount of resources required to check the password. The work factor // increases exponentially, so each increment is twice as much work. If // omitted, a default of 10 is used. string hashed = BCrypt.HashPassword(password, BCrypt.GenerateSalt(12)); // Check the password. bool matches = BCrypt.CheckPassword(candidate, hashed);
The source code is available via the links below. You can download the packaged version, which includes an NUnit-based test suite, or download the source directly via
BCrypt.cs.Attachments
- Tuesday, September 11, 2007
A Better .NET Regular Expression Tester
Because the only other online tool I could find for testing .NET regular expressions was slow and covered with ads, I decided to write a simple AJAX regular expression tester. It's certainly not fancy, but it works for me.
- Saturday, March 24, 2007
Emulating Vista's User Directory Structure on XP
Perhaps the best new "feature" of Windows Vista is the Unix-inspired re-organization of user home directories. That is, instead of placing user-specific files under the abhorrent
Documents and Settingsdirectory, each user's folder resides under the command-line-friendlyUsersdirectory in the root system drive. Even better, your music is no longer considered a document. It's hard to argue against the fact thatC:\Users\Derek\Musicis cleaner thanC:\Documents and Settings\Derek\My Documents\My Music(or in some cases, the 8.3 friendlyC:\DOCUME~1\Derek\MYDOCU~1\MYMUSI~1, ack!).I use Vista on a couple of my computers, and I've grown to appreciate the new folder structure. Gone are the days of hacking together your own home directory structures just to try and cope with the unfriendly madness (who hasn't created a folder or two under
C:\just to make them typable?). Unfortunately, I still use several machines running XP, so it's hard to rely on this sane structure -- leading to grotesque hacks such as functions calledghetto-dosifyin my.emacsfile (I wish I were kidding).There are some additional advantages to utilizing the built-in home directory structure rather than creating your own. For example, the Run dialog resolves path reference relative to your home directory. So, if you type
Start -> Run -> ., your home directory will open. Sub-folders can be accessed just as easily -- typingStart -> Run -> dev\blogopens my blog source tree. Also, many tools (backup, indexing, etc.) will only consider content in your actual profile directory (the one that Windows knows about).Another nice benefit is that the compact path structure is Cygwin-friendly, so you can safely set your
HOMEenvironment variable (also used by Emacs, of course). Thus,~/Desktopin Cygwin matches~\Desktopin Powershell. I keep myHOMEdirectory synced across many machines (Vista, XP, and Linux) using Subversion, and it's nice being able to keep everything in the same place.Fortunately, we can make some relatively simple tweaks to our XP systems to make the structure more closely resemble that of our Vista and Unix friends.
Junction, Junction, What's your Function?
Chances are you've been running your operating system for a while, and you've got a bunch of clutter under your
Documents and Settingsfolder. More importantly, you probably have a bunch of applications with stored absolute paths under that folder, not to mention all of those common dialog pointers. So obviously, just renaming theDocuments and Settingsfolder is going to cause more problems than it'll solve.If XP had symlinks (another feature added in Vista), of course, we'd have an easy solution. We could simply type
ln -s DOCUME~1 Users, and we'd be off to a good start. We could update environment variables to use our new path structure, and we wouldn't break the old path references. Fortunately, XP does support directory symlinks ... well, sort of. XP's default file system, NTFS, supports something called a "junction point". For our needs, it'll do.While junctions are supported by NTFS, there is no built-in tool for creating them. Enter Junction, a simple command line tool for creating (and viewing targets of) NTFS junction points. With the tool located in your PATH, run the following command sequence to create the link:
C:\Documents and Settings\Derek>cd \ C:\>junction Users "Documents and Settings" Junction v1.04 - Windows junction creator and reparse point viewer Copyright (C) 2000-2005 Mark Russinovich Systems Internals - http://www.sysinternals.com Created: C:\Users Targetted at: C:\Documents and Settings
Special Folders
At this point, we can access our home directory under the
Userspath, but unfortunately Windows has no idea that we've actually intended to change our home directory. For example, starting a fresh command line session stills shows that old, annoyingDocuments and Settingspath.To fix this, we need to dig into the registry. Under
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList, there should be a number of keys in the formS-x-x-xx-.... Browse through the keys until you find the one with aProfileImagePathvalue ending with your user name (e.g.%SystemDrive%\Documents and Settings\Derek). Change this value to use the new path (e.g.%SystemDrive%\Users\Derek). You will need to restart Windows for it to pick up the change.
After rebooting, you should default to the new
Userspath when starting a fresh console session. And because it's actually a junction point, all of your existing files should be right there with you. This is a nice start, but there are still some annoyances:My Documentsshould beDocuments,My Music(and its similarly named friends) should also drop the silly "My" prefix, and ascend one directory (unless you think music is a document, of course).Again, junction points are exactly what we need. Use the following command sequence to set them up:
C:\Users\Derek>junction Documents "My Documents" ... C:\Users\Derek>for %f in (Music Pictures Videos) do junction %f "My Documents\My %f" ...
You might also want to hide the
My Documentsfolder, so you don't see two references to the same folder when browsing your home directory. You could just make it hidden, but if you're like me you have hidden folder display enabled. In this case, we just need to make it hidden and give it the system attribute using the following command:attrib +S +H "My Documents".Our paths are now much prettier, but we do get stuck with the ugly default folder icon. Fortunately, you can right click on the folder whose icon you wish to update, click Properties, and change the icon in the Customize tab (you must do this before the next step, where they are assigned as special folders). I use the normal icons in
shell32.dllforDocumentsand the like. For my root home directory (Derek), I useuser-home.ico(attached to this post) from the Tango Desktop Project. The icon was conveniently translated to ICO format by Ben Brown. You might receive a warning about "enabling task folders", this is safe to ignore.As with the
Usersjunction point, we need to tell Windows about our new paths. For this, we'll use a Microsoft tool called Tweak UI. Install the tool, start it (Start -> Run -> tweakui), and select theSpecial Foldersitem underMy Computerin the tree control. Here you can select each path for which we created a junction, and assign the new path.
Common Dialogs
We've solved our ugly path problem, but overall usability still leaves something to be desired. For example, our common dialogs all have direct pointers to our
Documentsfolder, but it's still rather painful to get to our root home directory to, say, open a music file.Fortunately, Tweak UI comes in handy here as well. Under
Common Dialogs, selectPlaces Bar, and chooseCustom Places Bar. Here, you can define up to five paths that you will have easy access to using the common file dialog used by most Windows applications.
Here's the new dialog in action:
Another nice convenience is having a home directory link in your start menu, as below (simply drag the link there to create the shortcut):
Attachments
- Saturday, February 24, 2007
Screencast: Formatting a CSS File with Emacs
When I wrote The Case for Emacs, the main point I was attempting to convey was that Emacs is an amazingly effective editor even without the customizations it's so famous for. Right out of the box, you can do some pretty incredible things with its broad set of built-in commands. And if a simple Emacs configuration is good enough for Donald Knuth, it's good enough for us, right?
The other day I came across a CSS file that was in need of some formatting tweaks, so I slapped together a quick macro and fixed it up. Nothing special, but it got me thinking -- why not take this example to an extreme, creating the ugliest CSS file of all time, and create a screencast of cleaning the file up using a bare bones Emacs configuration? Well, here we are.
The CSS file you see getting the Emacs treatment in the screencast is not real. I intentionally created about the ugliest file I could, butchering indentation, casing, structure, mixing tabs and spaces ... you name it (the CSS file is attached, if you're especially curious).
I'm not necessarily suggesting that Emacs is the best tool for the specific task we're performing here, but it's a pretty broadly understood file format, so I thought it would be interesting way to demonstrate some of Emacs core functionality. If your favorite CSS editor has a magical "auto-format" button, by all means use it -- it's better than this strategy to be sure, but it's a lot less flexible!
The Emacs instance you see in the screencast is extremely close to a stock distribution. I made the following modifications, in the interest of making it easier to see what's going on:
- Disabled the tool bar and menu bar.
- Loaded mwe-log-commands.el, for demonstrating the keys pressed.
- Enabled the
downcase-regioncommand, normally disabled. - Loaded a simple CSS mode for font locking.
- Changed the default font to a narrower version, Consolas.
On with the show.
Here's a breakdown of how we attacked the file.
- Converted tabs to spaces with
untabify. This is a good first step when encountering a file as hideous as this one. - Normalized all spacing by compressing multiple spaces and newlines to a single space, using a regular expression replacement.
- Added newlines after all open braces (
{) using a macro. Also usedjust-one-spacebefore the brace to make the spacing consistent. - Added newlines after all semi-colons, again using a macro. Also compressed space in front of the semi-colons, and added a space after the colon delimiting the property from the value.
- Used another macro for adding newlines before and
after the close braces. I added the extra spaces before using
delete-blank-linesbecause the spacing varied based on whether or not a trailing semi-colon was present. - Killed a few empty blocks, using
backward-paragraphto quickly navigate blocks. - Executed another macro to make the property name case consistent (made them all lower-case).
- Sorted properties by name within each block using a
macro to regionize the block and call
sort-lines. - Compressed expanded forms of margin and padding specification to a single line, using a multi-line regular expression replacement with a reference to a captured group in the replacement string.
- Performed a few simple manual cleanups, and updated the messaging.
Attachments
- Friday, January 26, 2007
Emacs Hack #3: Compile Emacs from CVS on Windows
In previous hacks, we learned how to install and configure stable binary builds of Emacs. While the stable version (currently 21) is the best version to run for most users, you may be brave or curious enough to try one of the newest pre-release versions, 22 or 23. Because there are no official binary builds of Emacs beyond version 21, you will either need to install it from an unofficial source, or compile it yourself. This hack covers the latter.
Get the Source
The first step in compiling your own version of Emacs is getting the source. It is sometimes possible to obtain gzipped archives of the Emacs source at a given point, but it's typically more convenient to grab the source from CVS (this also ensures that you're using the most up-to-date version of the given branch).
If you don't already have CVS installed, fetch a recent version of
cvs.exefrom its distribution site, and place it somewhere in your path.Next up, we pay a visit to Savannah, the GNU development site. The Emacs project page contains details on how to obtain the sources. At the time of this writing, the following command will download the HEAD version of the code (Emacs 22).
cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/emacs co emacs
If you wish to build a version of Emacs other than 22, you will need to pass a tag name to your checkout command (or a subsequent update command). For example, adding
-r EMACS_21_3would check out the code for the Emacs 21.3 release (the stable release at the time of this writing). For a sneak preview of (the very unstable) Emacs 23, use-r emacs-unicode-2.After checking out the sources,
cdto thentdirectory. As a safety measure, issue acvs up -kbcommand here to make sure that all files in the directory have proper line endings. At this point, the source tree is ready to be built.Prerequisites
The Emacs build process requires a handful of tools that probably don't already exist on your system. These include GNU versions of tools with Windows equivalents (
cp,rm), as well as tools that are often unique to a GNU system (makeinfo). If you want your build to support images, you will also need a variety of libraries for rendering image formats. All of these tools are available from GnuWin32, a project which provides native Windows binaries of GNU tools.The packages you will need to install are listed below. In each case, I suggest installing the latest "setup" package, which will run an installer and place the binaries in a consistent location.
- CoreUtils
- This package contains a variety of tools, and is
required in order to build Emacs. Specifically, it contains
cpandrm. - TexInfo
- This package contains
makeinfo, which generates Info documentation from the texinfo sources in CVS. While technically optional, I strongly recommend installing it.
If you wish to build Emacs with image support (optional), you will also need the following image libraries (from the GnuWin32 Packages page:
At the time of this writing, the Xpm library is missing a required header file (
simx.h). You can either get it from the source package, or download it from the attachments in this post. This file should be placed in the GnuWin32includedirectory.Compiling with MinGW (GCC)
The simplest way to build Emacs is using MinGW, a collection of freely available tools for building native Windows binaries. The package includes the GNU C compiler, a port of
make, and various header files (including those from the Win32 API, such aswindows.h).The first step, of course, is to obtain the MinGW distribution if you don't already have it installed. The current version at the time of this writing is 5.1.3, and is downloadable at SourceForge.net (via the project page). Install to the location of your choosing, being sure to select the following components:
- MinGW base tools
- MinGW Make
After installing MinGW, add its
bindirectory to your path using a normal Windows Command Prompt session (set PATH=%PATH%;C:\MinGW\bin). Next, runconfigure.batfrom thentdirectory as follows to build aMakefile:configure.bat --no-debug --with-gcc
If you installed the libraries for image support, you will also need to pass the appropriate include path to the configuration script. If you installed to a directory with a space (the default), use the DOS name of the directory as demonstrated below:
configure.bat --cflags -IC:\Progra~1\GnuWin32\include --no-debug --with-gcc
If all goes well, you'll get a message telling you to run
gmaketo build Emacs. We're not quite ready for that yet, however. Because we got our source from CVS, we need to perform a "bootstrap" build. This creates a bootstrap Emacs binary to build autoloads and byte compile the Elisp (.el) files in the distribution. Begin the build process as below (mingw32-makeis the MinGW name forgmake):mingw32-make bootstrap
After the bootstrap completes, you're ready to build the source code. You'll also want to build the info files (make sure
makeinfo.exeis in your path) before installing. Run the following commands in sequence to finish compiling and install your Emacs build.mingw32-make info mingw32-make mingw32-make install
By default, Emacs will be installed "in place". If you prefer to install it somewhere else, run
configure.batwith a--prefix <dir>argument pointing to your preferred installation directory.Compiling with MSVC
The most popular C/C++ compiler for Windows is MSVC, Microsoft's Visual C++ compiler. Traditionally, this compiler was only available commercially, when purchased as part of Microsoft's Visual Studio development product. In recent years, however, Microsoft has made simple versions of the compiler available free of charge. Fortunately, the simple versions suffice for building Emacs.
At the time of this writing, the VC8 compiler (part of Visual Studio 2005) is the most recent version available. Unfortunately, Emacs cannot currently built with this version of the compiler without modifying the source. The only other freely available version of the MSVC compiler was made available as part of the Visual C++ Toolkit 2003. Microsoft no longer distributes this compiler (it was replaced by Visual C++ 2005 Express Edition), but if you already have it installed, or happen to have downloaded it previously (the file name is
VCToolkitSetup.exe), you can proceed to build Emacs using this compiler. If you have a commercial version of Visual Studio 2003 installed, the steps should be very similar.The Visual C++ Toolkit installation is very basic. In order to build Emacs, you need a variety of header files in addition to those packaged with the toolkit. To obtain these, you will need to download and install a (pre-Vista) version of Microsoft's Platform SDK. Because you only need a base set of components, the Web Install method is probably optimal. The following screen shot shows the components you need to have selected:
Next, add
cl.exeto your PATH in a normal Windows Command Prompt window. The easiest way to do this is to run thevcvars32.batscript packaged with the download, e.g.:"%ProgramFiles%\Microsoft Visual C++ Toolkit 2003\vcvars32.bat"
Next, you need to add the Platform SDK paths to your environment. For example:
set SDKROOT=%ProgramFiles%\Microsoft Platform SDK for Windows Server 2003 R2 set INCLUDE=%INCLUDE%;%SDKROOT%\Include set LIB=%LIB%;%SDKROOT%\Lib
If you are building with image support, you'll also need to add the GnuWin32 paths:
set INCLUDE=%INCLUDE%;%ProgramFiles%\GnuWin32\include set LIB=%LIB%;%ProgramFiles%\GnuWin32\lib
Normally we'd be done at this point, but we're still missing a couple of pieces. The first is
setargv.obj. Fortunately it's only missing in binary form -- it can be generated from the Platform SDK's CRT source directory. To build the required file and place it in the Platform SDK's lib directory, run the following command:cl /c /D_CRTBLD /I"%SDKROOT%\src\crt" /Fo"%SDKROOT%\lib\setargv.obj" \ "%SDKROOT%\src\crt\setargv.cWe're also missing a few essential binaries the build process expects:
nmake.exe,rc.exe, andlib.exe. All of these files are packaged with commercial versions of Visual Studio, but you'll need to find them elsewhere if you're using the Toolkit. The first two can be obtained free of charge by installing the .NET Framework SDK. Make sure you add itsbindirectory to your path after installing.lib.exeis simply an alias forlink /lib, so a simple batch file (lib.bat) will suffice. A sample is attached to this post -- place it somewhere in your PATH.Finally, we're ready to build Emacs using MSVC. As with the MinGW build, the process begins with running
configure.bat:configure.bat --no-debug --with-msvc
Things are pretty straighforward from here. As with the MinGW build steps, we need to start with a bootstrap build, after which we can build the complete distribution. Refer to the MinGW steps for details:
nmake bootstrap nmake info nmake nmake install
Optimized Builds
One of the advantages of compiling from source yourself is that you can optimize builds for your platform. For example, if you're using a modern AMD or Intel processor with SSE extensions, you can enable the compiler to generate optimized code for your processor. To build an optimized version of Emacs, you will need to pass arguments to the compiler by way of
configure.bat's--cflagsargument. To build a version of Emacs optimized for an SSE2-capable Athlon or Pentium 4 processor with MSVC, for example, you would run the script as follows:configure.bat --no-debug --with-msvc --cflags /O2 --cflags /G7 --cflags /arch:SSE2
For a similarly optimized build with MinGW (GCC), use:
configure.bat --no-debug --with-gcc --cflags -msse2 --cflags -O3
Optimized builds will typically be incompatible with older processors that do not support the selected extensions (in most cases, they will crash at runtime). If you are creating an Emacs build to share, minimize optimizations that require specific processor features such as SSE.
Final Steps
If you built Emacs with image support, you need to copy the runtime DLLs to Emacs'
binpath (or your system path, if compiling only for a single system). You can copy them manually, or script it as below (specific file names may vary slightly if versions change):for %f in (giflib4 jpeg62 libpng13 libtiff3 xpm4 zlib1) do \ echo copy "%ProgramFiles%\GnuWin32\bin\%f.dll" ..\binAttachments
- Friday, January 05, 2007
Emacs Hack #2: Manage Emacs Instances with gnuserv
If you followed Hack 1, you can now launch Emacs and open files from within the interface. On Windows (and in some other environments), you can also drag files to Emacs from your window manager.
But what if you want to send a file to Emacs from a console session, or via a shortcut? You could invoke Emacs using
runemacs(or create a shortcut to it), but every time you do that you end up with a whole extra instance of Emacs -- a unique operating system process which is independent from any previous instance(s).In some cases, this might be what you want. In the rare case of an Emacs crash, for example, you would not lose any active buffers in the other instances (note that Emacs' auto-save facility would probably mitigate the damage here). Most of the time, however, you'd rather open the file in an existing instance of Emacs. This has a number of advantages:
- Memory is conserved by sharing a single process.
- Shortened start-up time since Emacs is already loaded.
- All open buffers in the shared instance can be quickly accessed from any Emacs frame.
- The buffer list can be used to see every open file across frames, and to perform actions on them.
- Dynamic abbreviations (covered in a future hack) can be sourced from a larger set of files.
- Contention issues (multiple processes accessing the same file) are avoided.
Fortunately, there is a small client / server program called gnuserv which enables us to do just this.
Installing
Installing gnuserv consists of two parts -- some platform-specific binaries (
gnuserv,gnudoit,gnuclientand, on Windows,gnuclientw), and an Emacs Lisp file which we'll load into Emacs. The Lisp code will spawn thegnuservprocess after Emacs has started, listening for received commands ongnuserv's standard output stream.On Windows, the first step is to unpack the ZIP file containing the gnuserv binaries (on Unix and derivatives, simply install the gnuserv package for your distribution). The Windows port is available at Guy Gascoigne - Piggford's site, or attached to this post if the site is unavailable. Extract the binaries to a directory in your
%PATH%(see Hack 1 for some tips on setting up your path), e.g.C:\Program Files\gnuserv.Next, we need to install the Emacs Lisp portion of gnuserv. This involves placing the
gnuserv.elfile (attached to this post) somewhere in Emacs' load path. The easiest way to do this is to copy the file to thesite-lispdirectory, typically located under your base Emacs installation directory (Windows) or at/usr/local/share/emacs/site-lisp(Unix and its derivatives). Files placed here are automatically available to Emacs.Configuring
We've installed all of the pieces necessary for gnuserv, but nothing has really changed in terms of Emacs' behavior. Even though we've added
gnuserv.elto its load path, we still need to instruct Emacs to load the file. We'll do this by adding some initialization code to our.emacsfile (see Hack 1).From within Emacs, type
C-x C-f(that's the control key plus x, followed by the control key plus f -- you can keep the control key pressed the whole time). This invokes Emacs'find-filefunction, which prompts you for a file path to open using the minibuffer (the bottom line in the frame). Emacs will present you with a suggested path. Type~/.emacs, pressTab(Emacs will remove the suggested path as part of its completion algorithm), and then pressEnterto open the file.If you're a Windows user, this path might look strange. The tilde (
~) character is simply a handy shortcut for the folder represented by your%HOME%environment variable, and the forward slash is Emacs' preferred path separator (that is,C:/Usersis preferred toC:\Users). The path syntax for the home directory is not only convenient to type, it also makes your Emacs configuration more portable. For example,~/Desktopalways maps to your Windows desktop, regardless of your login name or other machine-specific details.At this point, you should be staring at a blank buffer. Add the following lines to your
.emacsfile:(require 'gnuserv) (gnuserv-start) (setq gnuserv-frame (selected-frame))
This might look somewhat strange -- it's actually just some simple Elisp which tells Emacs to load and start gnuserv. The first line calls the built-in
requirefunction, which tells Emacs to load gnuserv if it hasn't already done so (this causes Emacs to go findgnuserv.elin our load path). The next line calls a function ingnuserv.el, which starts (or restarts) thegnuservprocess. The last line is an optional customization which tells Emacs that it should open new files from the clients of gnuserv in its startup frame (if you find that you prefer to have multiple Emacs frames, comment this line out by prefixing it with a;character, or delete it entirely).Emacs reads its
.emacsfile and evaluates these lines each time it starts up. You can either restart Emacs now, or pressM-x(M stands for meta, which is typicallyAlt), typeeval-buffer, and pressEnter(this tells Emacs to evaluate the contents of the current buffer without requiring a restart).Launching Emacs
At this point, you should have an Emacs instance running with
gnuservstarted. So far, so good. Now it's time to use the client programs which came with the gnuserv distribution.gnuclient and gnuclientw
The
gnuclientprogram can be used to open a file in a running instance of Emacs. For example, typinggnuclient READMEfrom a console session will open a new buffer for the fileREADME(even if it doesn't yet exist). If Emacs is not currently running, it will automatically be started (at which point it will evaluate its.emacsfile, loading gnuserv).You may notice that when a file is opened using
gnuclient, a message is displayed in Emacs' minibuffer: "When done with a buffer, type C-x #.". You may also have noticed that when you invokegnuclient, it does not immediately exit (that is, you can't continue using the console). What's happening is that thegnuclientprocess is waiting for you to finish working on the file -- when you pressC-x #from within Emacs, thegnuclientprocess exits. This is convenient in many cases -- for example, when editing a Subversion commit message. If the process returned immediately, Subversion would not be able to read the message you typed in to Emacs because there is no longer any connection between the editor and the launcher.In other cases, however, you just want to open a file in Emacs and continue using your console session immediately. In this case, you can use
gnuclientw(Windows only -- on other platforms, usegnuclient -q). This is a variation ofgnuclientwhich returns immediately. It's also compiled as a Windows application, so it's ideal for creating Windows shortcuts (gnuclientopens an unneeded console window in these cases). I typically add a shortcut tognuclientwin%USERPROFILE%\SendToand%USERPROFILE%\Desktop. In practice, I nearly always usegnuclientw(with or without arguments) to start Emacs on Windows (because it's in your path, it can also be executed from the Run dialog). If you find yourself runninggnuclientwa lot when Emacs is already open, try adding a-xoption to "top" the Emacs frame (make it visible if it's hidden).gnudoit
The last (and in this case, the least) client program is
gnudoit, which can be used to evaluate an Elisp form. For example,gnudoit (list-buffers)will open a window within your Emacs frame listing the currently open buffers. This command is deprecated because the same behavior can be achieved usinggnuclient-- on Windows, usegnuclient -e (list-buffers). On other platforms, typegnuclient -batch -eval (list-buffers). In practice, you probably won't be running these forms very often.Attachments
- Saturday, December 09, 2006
Emacs Hack #1: Install Emacs on Windows
Installing Emacs on most platforms is a common and well supported operation. On Linux, for example, it's typically installed via the package management system for the particular distribution you've chosen. This hack covers installing Emacs on Windows, where it's a bit more challenging.
There are some shortcuts to getting Emacs installed and running on Windows, but we're going to walk through the steps from scratch. There are a number of "decision points" throughout the installation. When we reach those, I'll give a specific recommendation as well as provide you with the alternate options.
Which Emacs?
The first decision greets us before we even embark on the installation process. There are two primary flavors of Emacs -- GNU Emacs, and XEmacs. The former is the "original" Emacs, and the latter was branched from an earlier version of GNU Emacs by a company called Lucid. If you're interested in the history, Wikipedia has a nice summary.
I've used both versions extensively and had similarly positive experiences with each. I'm going to recommend installing GNU Emacs, however, since in my experience it seems to be the more popular version. Because it's more common, it's more likely that additional packages and snippets that you adopt after the installation will be more compatible. Future posts in this "hacks" series will also assume that you're running a flavor of GNU Emacs.
OK, so we've decided to install GNU Emacs. However, we're not quite finished yet. There are several versions and packagings of Emacs (yes, even when we limit our target platform to Windows), and so we've got yet another decision to make. Currently, the "stable" version which is distributed by GNU officially is version 21.x. The next version of Emacs, 22.x (and the version after that, 23.x), can be had by compiling your own version from CVS (there are also unofficial binary builds).
I'm going to walk you through the steps of installing the official, stable distribution of Emacs (21.x). If a newer version is distributed by the time you end up reading this, the steps are likely to be identical. Many of these steps will also be relevant for unofficial distributions, so it's a good place to start even if that's where you end up.
Before we bother to even find the official binary distribution of Emacs, though, we need to make sure our Windows environment is set up properly.
Home is Where the Files Are
As you use Emacs, it will occasionally need to read and write files to your hard drive to keep track of various settings and customizations (we'll cover many of these in this "hacks" series). The primary customizations are placed in a file called
.emacs(the.preceding the name causes the file to be "hidden" on Unix file systems). On Unix and Linux systems, this file is placed in the "home" directory, a user-specific location denoted by theHOMEenvironment variable (e.g./users/derek).Windows does not set the
HOMEenvironment variable by default, so Emacs assumes that your "home" directory isC:\. This is generally not a good place for user-specific settings. On most Windows systems, you need administrative privileges to write to the root directory. Also, multiple users on a given Windows system would likely want their own configurations. Lastly, many Windows backup and migration tools only save the contents of user-specific directories.Fortunately, it's a simple matter to tell Emacs where our real "home" directory is -- we simply need to set the
HOMEenvironment variable. I strongly suggest using the value of theUSERPROFILEvariable as the basis for settingHOME. This variable is always available and is properly set to the real "home" directory of the current user.We can temporarily set environment variables from the console using the
setcommand, but we need to make this particular variable persistent. If you're using Windows Vista, you can set persistent environment variables using thesetxcommand (you can also do this on Windows XP if you install the necessary support tools). To setHOMEfrom a normal user console session usingsetx, type:setx HOME "%USERPROFILE%"
If you don't have the
setxcommand, you can set this variable using a graphical tool. ChooseSystemfrom the Control Panel menu, click theAdvancedtab, and press theEnvironment Variablesbutton. In theUser variablessection, add a new environment variable with the nameHOMEand the value set to your user profile directory (e.g.C:\Documents and Settings\Derek). You can typeecho %USERPROFILE%from a command prompt to get the correct value.
To test the value, start a new console session, and type
echo %HOME%. On Windows XP, for example, you should see something like this:Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\Documents and Settings\Derek>echo %HOME% C:\Documents and Settings\Derek
Getting the Distribution
With that out of the way, we can download Emacs. Begin by visiting the official Windows distribution site. You'll notice a number of different files here:
emacs-21.x-barebin-i386.tar.gz- This contains Emacs binaries without any precompiled Lisp files. You could install Emacs with this file, but it wouldn't be very useful (you would have to add / compile the Lisp files yourself to make Emacs useful).
emacs-21.x-bin-i386.tar.gz- The is a full installation of Emacs with precompiled Lisp files. This is a good choice if you just want to install Emacs and use it.
emacs-21.x-fullbin-i386.tar.gz- This is functionally the same as the previous item, but includes the full Lisp source in addition to the compiled versions. This is the best choice if you want to be able to see the actual code behind much of Emacs. If you have the space, I recommend installing this version.
emacs-21.x-leim.tar.gz- This is the "Library of Emacs Input Methods" package, which is used for entering non-ASCII characters. You can probably skip this for now.
emacs-21.x-lisp.tar.gz- This file contains the Lisp source for the Emacs distribution. If you installed the regular binary version, you could use this to turn it into the "fullbin" version.
emacs-21.x-undumped-i386.tar.gz- This version contains a special executable you can use to rebuild Emacs after changing built-in files. You almost certainly don't need this.
fns-21.x.x.el- This file contains the load history for built-in libraries. This file comes with the normal distributions, so you can safely ignore it.
The installation process is the same whether you install "bin" or "fullbin", so download whichever option makes more sense for you.
Unpacking
Because the files are in gzipped GNU Tar format, you cannot extract them using the built-in compression tool in Windows. If you already own a commercial compression tool such as WinZip or WinRAR, you can easily extract the binaries using one of those tools. If you don't have one of these tools, I recommend installing 7-Zip, which has similar capabilities and is completely free.
The root of the binary distribution contains a single directory,
emacs-21.x. Because there are no user-specific files in the binary distribution, it makes sense to install it in a shared location. If you have administrative privileges, I suggest extracting it to your "Program Files" directory (typeecho %ProgramFiles%from a command prompt to get the full path). Users with no administrative privileges whatsoever can install Emacs in theirHOMEdirectory instead.If you're using 7-Zip, open it and navigate to the downloaded file. If you're using Windows Vista or are running as limited user account, you will need to start 7-Zip using the "Run As" feature of your particular flavor of Windows. If you don't, it won't have access to write to the shared directory. Once you've opened the downloaded file using 7-Zip, double-click
[Content], select theemacs-21.xdirectory, and clickExtract. Enter the full path to your "Program Files" directory, and click "OK".
You can now close 7-Zip, navigate to your "Program Files" directory and confirm that the extraction was successful.
Installing
You now have a usable version of Emacs. If you execute
runemacs.exefrom thebindirectory, Emacs will open and be fully functional. This isn't the most convenient way of starting Emacs, however. Thus, there is one last installation step.In the
bindirectory containing therunemacs.exefile, there is a file calledaddpm.exe. This file "installs" Emacs by adding a start menu program group and adding a few registry entries inHKEY_LOCAL_MACHINE\SOFTWARE\GNU\Emacs. Simply double-click the file, click "OK", and Emacs will be fully installed. Now you can open Emacs from the "Gnu Emacs" program group in your start menu.Now What?
Now that Emacs is installed, feel free to start using it. If you're accustomed to Windows editors, chances are it will feel a bit strange at first. Perhaps the most useful thing you can do at this point is to invoke the Emacs Tutorial (click the
Helpmenu item to access it). This will cover some of the basic Emacs commands and concepts. If you complete the entire tutorial, you should start to feel comfortable enough in Emacs to do some basic editing. Practice using the Emacs commands as much as you can afford to. You'll learn hundreds of them over time, but the commands covered in the tutorial are some of the most important ones. - Sunday, December 03, 2006
The Case for Emacs
When well-seasoned programmers get together to speak of the "good old days", there are a few persistent topics. There are always at least a couple of archaic programming languages ... depending on the number of reminiscing participants, there can even be enough to inspire a game of "I had it worse than you did!". Inevitably, this is followed by some form of reference to ancient multi-user operating systems, mainframes, or -- although increasingly rare -- systems involving punched cards. More often then not, this piece of the conversation will lead to discussions of operating ancient editors via terminals -- with ed nearly always starting that conversation (and earning several grimaces at its first mention).
If you've heard enough of these discussions, as I have, you've surely observed that the tone of these discussions suggests that we are now in a golden age of development tools ... I've been writing software, and hearing these discussions just long enough to start to wonder if these discussions were any different back then (it's all relative, isn't it?). Me? Well, I started this game a bit late relative to most of the "good old days" participants I've observed. They had Pascal, I had VB5 -- they had ed, I had ... well, VB5 ... they had libraries, I had BBSes, they had printed manuals, I had CD documentation ... the list goes on.
The Beginning
I stumbled into programming during high school (I'm excluding the
10 print "derek", 20 goto 10fun that so entertained me in grade school). I was responsible for inputting data penned in by customers entering contests, signing up for mailing lists, etc. into a database. Every time I re-entered the same postal code three times in a row I could feel my brain shrinking.Within the friendly environment that was VBA (in Microsoft Access '95), I managed to script a form into filling in a few fields for me when encountered with one of the most popular postal codes. I was hooked -- entering the addresses had been transformed from mind-numbing boringness to a creative exercise in trying so save as many keystrokes as possible.
My initial fondness for VBA (and Microsoft's influence on programming curricula in the greater Puget Sound area) led me to a Visual Basic programming course at a local community college. I enjoyed it, and it eventually led me to my first programming job (which really was a web design job, but nobody seemed to mind -- perhaps because I was still in high school, or perhaps because I was making $8/hour).
In these early years of my career (before it was officially my career, I should add), there was never a distinction between language, framework or development environment. My language was Visual Basic, I called Visual Basic functions, and I did all of this using a product called Visual Basic ... the explosion of Active Server Pages brought Visual InterDev along for the ride, and C++ tasks were handled by Visual C++. There was always a "right" tool for the job.
A Hot Cup of Java
After a couple of years of making ASPaghetti, I started toying with a language called Java. I use the word "toying" quite intentionally, because at the time my best description of it would have been "language used to make cool buttons on Web pages". I had been quite ignorant of the growing Java juggernaut, but our company had recently bought some Oracle licenses (it was the Dot-Com era, why not?) and it seemed to be the language of choice for folks in that camp.
I quickly moved from "toying" to "using". For a Visual Basic programmer, Java was like heaven -- actually, Java was heaven. After two weeks of using it, I was ready to have the "good old days" conversation about old Visual Basic. Goodbye class modules, hello beans!
One thing that was very different in the Java world, and it was new to me, was choice in environment. In the Microsoft world (and this is still true today), there was a supported development environment released with the language ... in Java, the tutorials were telling me to set a CLASSPATH environment variable (bad advice, by the way --
-cpis much less painful, but I digress) and use the editor of my choice. Using Notepad was fine for my "First Cup of Java", but I knew I'd be needing auto-completion and a "debug toolbar" before long.My quest for the perfect Java IDE never really finished -- I started with Oracle JDeveloper (basically an early version of JBuilder), and later dabbled with VisualAge, Webgain, Netbeans, JBuilder itself, and just about everything else that came around ... I never fell in love with any of them, but most were a pretty nice step up from the development tools I had worked with previously. They all had the basic features I was looking for -- syntax highlighting, intelligent completion, click-to-run, debugging ... none of them stood out, but they were all Good Enough[tm].
The Dark Side
There is a dark side of the programming world. Some are never exposed to it ... some have only had enough exposure to be scared by it. The dark side consists of a wide network of programmers, about 1 to 2 for every 10, who throw away the rules. They run strange operating systems, they spend free time on weekends experimenting with programming languages you've never heard of (Hask-what? Eif-who?) ... and they don't use the same tools that you do, to say the least.
I remember the first time I witnessed a member of this "dark side" in action. I had wandered into the cubicle of one of our recent hires, a long-haired "Unix guy" who was in the midst of a rather furious coding session in a very strange looking editor (Vim, for the curious). Watching him operate his editor was truly an experience. He was working at a pace I'd never seen before, his cursor flying around the buffer like a pinball. I was like a child seeing a magician for the first time (that hat was empty!).
I rudely interrupted his session -- in part to marvel at his proficiency, but also to ask some questions about his current project. He walked me through the code he was working on, and the artistry continued -- he seemed able to will his cursor around the buffer with the power of his stare, as if he was simply "thinking it" there. I'd never seen anything like it -- he wasn't using his editor -- he was his editor.
I was forever changed. I had seen what the dark side was capable of, and I wanted a membership card. Going back to my IDE was a depressing affair -- I tried -- and I tried hard, to replicate the genius I'd just seen. He had clued me in to a couple of his tricks, which I tried to emulate. I took a tour through my IDE playground again, trying to find one that would enable me to perform the magic tricks he executed so effortlessly, but I couldn't.
I spent a week or so trying to convince myself that all of that "other stuff" in my IDE made up for what it lacked in the editing department. I was partially successful -- I mean, my debugger sure looked a lot fancier than his (gdb, for the still curious). That had to count for something. And that intelligent completion -- surely I couldn't live without that, right? Right? Not exactly ...
I made a conscious effort that week to think about each feature as I used it. Intelligent completion? Sure, it was helping me code, but how much? With this in mind, I noticed that I was doing some really idiotic things, like typing
n-eand then waiting a (brief) moment for the completion window, in some cases using my arrow keys to find the appropriate completion, and selecting it. This, as opposed to just finishing the keyword ...x-t... surely, somebody observing from the dark side would be shaking their head at the sight of this. I had to do better. My tools were good, maybe even great, but they still weren't good enough.I Think I Can ...
With the help of a Unix shell account, I decided to take a tour of the dark side. I did a bit of research (Have you heard of this fantastic new search engine called Google? No, no -- l-e ... yeah, that's it -- now search for "cigar!"), and found that the dark side was in the midst of a civil war -- the Emacs-folk vs. the Vi(m) folk. Oh boy ... I was going to have to choose. A bit more reading convinced me that what I was seeking (magical editor powers, that is) could be achieved with either. I'd seen what Vim could do, how about this Emacs thing?
My first experience with both editors was a disaster ... in Vi, I did finally stumble into the proper mode for getting characters into the buffer, but it wasn't a pretty process (and it certainly wasn't intentional). Emacs started out a little better, but I couldn't figure out for the life of me how to quit -- control-s? control-q? And yeah, the fact that I kept toggling flow control on the terminal certainly didn't help. Finally, I figured it out -- control-z! Pressing that makes Emacs go away, problem solved. Alright, time for some magic tricks! Oh, and it took a couple of hours before I realized that I was suspending it rather than quitting. When you're accustomed to control-q,
\C-x\C-cis a long ways away. Fortunately, the tutorial (\C-ht) got that out of the way in a hurry.Armed with some documentation and a bit more time, I ended up giving each editor a fair shake. In no time, I was moving around buffers pretty well in Emacs -- not exactly comfortable, but reaching for my arrow keys less and less (not because it made sense to me, but it had been suggested that this was the path to magic, so I figured I'd try). Vim was starting to become less painful as well, although the mode switching was really causing me to do some amazingly stupid things (something tells me Vim users all have rigorous backup schedules). in any case -- I figured that to become a magician I would need to focus on one editor, and my laziness led me to Emacs (sorry Vim, too much early mode pain).
Magic Tricks
It really didn't take too long to reach my goal -- I had to re-train a lot (OK, nearly all) of my editing instincts, but before long I was performing buffer magic tricks of my own. By this time, I'd found other "dark siders", and I was extracting as much information as I could from them ("Hey, how'd to swap those characters?") ... each tidbit I picked up, I would force myself to use for a day, and inevitably, I'd end up using it 100 times the next week, wondering how I ever lived without it.
That's really the Zen of Emacs. When I encounter people who see me using Emacs, they typically assume that I use it because of its customizability. That's a nice feature, but it's not the reason to use Emacs. The Zen part is all there in the base package -- even without a
.emacsfile (a Lisp file read at startup used to customize Emacs). The "magic tricks" that turn you into an editing wizard aren't really tricks at all -- they're core features. Once you learn these core features, it's astounding the magic you can do -- and you can do it everywhere. You can swap arguments in C the same way you swap words in a text file -- you can autofill your comments just like your paragraphs -- rectangle insertion makes block comments as well as it adds a shared prefix -- dynamic abbrevs (rich, but unintelligent completion) work everywhere -- and all of this stuff works whether you're editing C#, Javascript, CSS, Java, XML, Ruby, Lisp, ... the list goes on. And trust me, these features only sound exotic until you depend on them.That's not to say you can't add useful tricks on top of these core features. The point is, you don't have to. I think a lot of people avoid Emacs because of this misunderstanding. The truth is that Emacs isn't scary -- you don't need to know a thing about Lisp, Emacs Lisp, or Unix (Emacs works wonderfully as a native Windows binary). You don't need to have a
.emacsfile, you don't need a PHD in Parenthesis to be a magician. You do need to give it some time, but it will pay you back.Emacs doesn't seem like much at first. It takes an hour or two to get the basics down, at which point a new user can probably use it with almost-Notepad-level efficiency. It takes a lot more time to get to the next level, but once you do, you can't go back. I remember explaining to an early Emacs user how much I loved prefixed arguments (numeric arguments to alter command behavior). He laughed at me like I was joking. I mean, that's fair -- prefixed arguments do sound like a silly "border feature" if you've never worked with them before. And hey, it took me a while to pick up on them. But now, I use them many times daily.
Stretch Yourself
Another thing you get with Emacs is flexibility. Have you ever wanted to experiment with a new technology, but avoided it because you were too comfortable in your existing programming environment? Come on -- you saw that Ruby on Rails screencast, right? If you're an Emacs user, you can be writing C# on Windows Monday, Ruby on Mac OS Tuesday, and Python on Linux on Wednesday. In each case, there are language-specific tools to use, but the place you spend the most time -- your editor -- is consistent across tools and platforms. Virtually all the time you invested in learning (and customizing) the editor comes along for the ride each time. Emacs lowers the bar.
OK, I know -- you're not convinced. You love -- no, you love your IDE. I've been there. How about 5 years ago, though? Were you using the same IDE? Were you even using the same language? Me neither. How about in 5 years -- do you think you'll be using the same language? The same tool? Really? If you did switch, how much knowledge from your last development environment transferred? You might think that "it doesn't matter" -- heck, I probably would have said the same thing reading this 5 years ago ("bah, I'll just learn a new IDE when I pick that language up"). It's difficult to explain how much you can gain from mastering an editor -- you just have to experience it. Have you ever learned a keyboard shortcut in a program, and wondered how you ever lived without it? Gaining proficiency in Emacs is like that feeling times 1,000. Emacs is never having to press the same button 5 times in a row.
It's powerful, even if it doesn't sound like it. After several years of using Java, I needed to switch to C# for a few projects which integrated with native Windows libraries. My development process hardly changed in the process -- every trick I knew for editing, buffer management, compiling -- it all transferred. With the recent shift towards standards-based Web-centric applications, and working for a startup, I'm writing C#, JavaScript, Java, CSS, C++ and XML regularly -- all within the friendly confines of Emacs. And yes, I'm writing this blog post in Emacs (using a custom RELAX NG Compact schema in NXML mode, for the still curious).
Emacs Hacks
Getting started with Emacs can be a bit intimidating at first. It's a community run by dark-siders, and they have a tendency to assume you're a rocket scientist. Wading through the documentation and getting out of the gates is the hardest part (it was for me -- I'm not even a model rocket scientist). I'm going to dedicate a few posts on this blog (in "hacks" style, sans the book-length depth) to helping you do just that. I'll probably bias it somewhat towards installing and using Emacs on Windows, since it's quite a bit easier on Linux (
apt-get install emacs21doesn't make for a very interesting blog post).If you're not convinced -- I understand. And it doesn't bother me at all. I have nothing against language or environment-specific tools. I use them all the time -- debuggers, static analysis tools, inspectors, refactorers -- you name it. You don't have to dump your IDE -- you can even use them side-by-side if that's more comfortable for you. My point is that mastering a flexible editor (Emacs or otherwise) can make you a more versatile, more productive developer. It might even make you a better developer. No Elisp hacking required.
- Saturday, October 21, 2006
Channeling Ruby in C# 3.0
I happened across a posting earlier today which contrasted Java and Ruby code snippets. The intent of the post (an enjoyable read, by the way) was to demonstrate the elegance of Ruby's simple, compact syntax, using the more verbose Java as a counter-example.
This post, on the other hand, attempts to use some of C#'s more recent features to demonstrate how static languages are closing the "elegance" gap (for lack of a better term) which the original poster demonstrated.
Extension Methods
The author opens with this example:
10.times { print "ho" }The author of the post implies that this is possible because Ruby treats
10as an object, while Java does not. The same is true in C# (integers do not inherit fromObject, and are "primitive"); however, with the help of extension methods in C# 3.0 I can very closely emulate Ruby's syntax:10.Times(i => Console.Write("ho"));The integer here is never boxed -- this is just syntactic sugar for explicitly calling the extension method with an explicit argument. Here's the entire source of the extension method:
public static void Times(this int value, Action<int> action) { for (int i = 0; i < value; i++) { action(i); } }The extension method syntax is a C# 3.0 specific feature, but is callable from .NET 2.0 code (and in fact, runs on the 2.0 CLR) using the more explicit syntax:
RubyishExtensions.Times(10, delegate(int) { Console.Write("ho"); });The author lists several other examples which are easily replicated using C# 3.0 extension methods. Looking for an odd number?
if 11.odd? print "Odd!"
In C# 3.0:
if (11.Odd()) Console.Write("Odd!");The relevant extension method:
public static bool Odd(this int value) { return value % 2 != 0; }Counting bytes?
102.megabytes + 24.kbytes + 10.bytes
In C# 3.0:
102.Megabytes() + 24.Kilobytes() + 10.Bytes();
Three very simple extension methods help here:
public static int Megabytes(this int value) { return value * 1024 * 1024; } public static int Kilobytes(this int value) { return value * 1024; } public static int Bytes(this int value) { return value; }How about ordinalizing a number?
print "Currently in the #{2.ordinalize} trimester"Again, in C# 3.0:
Console.Write("Currently in the {0} trimester", 2.Ordinalize());This extension method is a little longer -- but, like all extension methods, it's reusable:
public static string Ordinalize(this int value) { string ordinalized; if ((value % 100).Between(11, 13)) { ordinalized = "th"; } else { switch (value % 10) { case 1: ordinalized = "st"; break; case 2: ordinalized = "nd"; break; case 3: ordinalized = "rd"; break; default: ordinalized = "th"; break; } } return value + ordinalized; } public static bool Between(this int value, int min, int max) { return value >= min && value <= max; }Our extension method fun ends with a few date examples:
puts "Running time: #{1.hour + 15.minutes + 10.seconds} seconds" 20.minutes.ago 20.minutes.until("2006-10-9 11:00:00".to_time)In C# 3.0:
Console.Write("Running time: {0} seconds", (1.Hours() + 15.Minutes() + 10.Seconds()).TotalSeconds); 20.Minutes().Ago(); 20.Minutes().Until(DateTime.Parse("2006-10-9 11:00:00"));Predictably, it's extension methods to the rescue again:
public static TimeSpan Hours(this int value) { return TimeSpan.FromHours(value); } public static TimeSpan Minutes(this int value) { return TimeSpan.FromMinutes(value); } public static TimeSpan Seconds(this int value) { return TimeSpan.FromSeconds(value); } public static DateTime Ago(this TimeSpan timeSpan) { return DateTime.Now - timeSpan; } public static DateTime Until(this TimeSpan timeSpan, DateTime target) { return target - timeSpan; }Properties
The next example demonstrates a feature of Ruby's syntax which I cannot emulate -- its tight property syntax:
class Circle attr_accessor :center, :radius end
C# is a bit lighter than Java, but it's more finger effort here:
class Circle { private Point center; private float radius; public Point Center { get { return this.center; } set { this.center = value; } } public float Radius { get { return this.radius; } set { this.radius = value; } } }Collections
The next section of the post contrasts the collection syntax ... the author argues in favor of Ruby on a number of points here, none of which I wholeheartedly agree on (I prefer to be explicit with collection choice and usage).
Initializer syntax is singled out for non-array types (fairly). This is another problem nicely solved in C# 3.0 by way of collection initializers. The following snippets both result in a strongly typed
intarray of 3 numbers, with the second array coming by way of a generic collection:new int[] { 1, 2, 3 }; new List<int> { 1, 2, 3 }.ToArray();The author next argues for a more complete collection of methods on various collections, including dynamic array growth. Some of this could be accomplished with further extension methods, but I personally feel that
PushandPopbelong onStack, notArray, so I'm not going to bother implementing those.In terms of dynamic array growth, that is handled quite nicely by collection classes which are optimized for that case (
List<T>et al). Array-like syntax is supported (as it has been since C# 1.0), but my personal preference is to call anAddmethod when I want to add something.File I/O and Regular Expressions
The last language-level feature the author contrasts is parsing sentences containing the work "Ruby" from a text file:
File.read('test.txt').scan(/.*?\. /).each { |s| puts s if s =~ /Ruby/ }This is compact, to be sure, but we don't need any C# 3.0 tricks to get close:
using (StreamReader reader = new StreamReader("test.txt")) { foreach (Match match in Regex.Matches(reader.ReadToEnd(), @"(.*?\.)\s+?")) { if (match.Value.Contains("Ruby")) { Console.WriteLine(match.Value); } } }Summary
The point of this post was not to imply that C# is better than Ruby, or Java for that matter. I simply wanted to point out that brevity and elegance are not the exclusive domain of dynamic languages. "Syntactic sugar" can go a long ways, without requiring fundamental changes to a language or its runtime. C# 3.0 is a particularly relevant example, given that it runs atop the 2.0 CLR despite all of its syntactic sugar.
If you're interested in toying around with the examples cited in the program listings above, I've attached the sample code to this post. To use it, install the LINQ CTP and run MSBuild from a command prompt.
Attachments
