Tag Archives: software development

Plotting Software, Pantsing Stories

A screenshot of some code I've been writing
Some code I’ve been writing

I just completed a demo1 for software that me and my team have been working on for nearly a year. It isn’t quite done yet—we still have a few months of work left, but it feels good to get to the point where you have something to show. Indeed, it’s not that different from getting a story out into the world.

When it comes to writing (when I can even manage to write these days) I am “pantser” as opposed to a plotter. That is, I don’t plan out my stories in detail. I have an idea of where the story is going, and I figure out how to get there as I write. Sometimes, I end up somewhere else entirely.

With software, I am the complete opposite. Over the decades I’ve worked on increasingly more complex software and I’ve found that my brain doesn’t have the capacity to build it without plotting it out first. I hadn’t really considered it much until now, but I suppose that designing software is a lot like outlining a story. You start with the high level goals and requirements; you identify the tools you can use to meet those requirements; and then you figure out how everything will fit together to make a cohesive and self-consistent thing.

The day-to-day work is writing code, small fragments, akin to scenes in a story or novel. Often you write something that gets the job done but isn’t particularly elegant, so you rewrite it and rewrite until it purrs. The world fades around me when I get into this mode, my focus is completely on the task at hand, and an 8- or 9-hour day can fly by in what seems like the blink of an eye. This happens when writing, too, but I can’t consistently write for 8- or 9-hours the way I can work on code.

I often had the impression (from comments I’ve heard in various places and times) that non-writers think that writing is easy. I find it difficult, and I’m fairly worn out if I manage to writer for more than 2 hours. Writing code is equally difficult, but I come away from these long stretches in what I call a “code-coma.” It’s hard to re-engage with the world around me, and I have to ease back in.

When I have something on the page that works, I’ll review it, often out of context of the rest of the piece (often because “the rest of the piece” doesn’t exist yet). I’ll find typos here and there, little misspellings, or autocorrects that don’t work. Same thing when writing code. Punctuation is just as important and I can stare at a small piece of code for an hour wondering why isn’t working, only to realize I was missing a semi-colon somewhere.

Sometimes, you get pretty far along in a story only to realize that you’ve uncovered a major hole in the plot. This happens with software, too, but it happens less frequently for me these days because I “outline” the software and try to eliminate plot holes in the design before we actually start building the thing. Still, other things may trigger major changes forcing rework.

When I think a story is ready (always after the second draft, never after the first) I’ll send it out for critique by fellow writers. It’s no different with code. We sit down for code reviews where it becomes our job to ask each other (and ourselves) tough questions about the choices we made. These are both incredibly useful and incredibly disheartening. Code is always improved in these reviews, but I find it disheartening that I couldn’t think of some of the elegant ways my colleagues suggested for improvement in the first place.

(In the movie business, I think the information that comes out code reviews would be akin to “notes.”)

The draft of a story that goes to the editor might be considered a beta (or a golden master) in the software world. It is almost good enough for the world, but subject matter experts (editors, copy editors, publicity people) will look at it and offer some final polishing suggestions.

Finally, the book or story is out in the world! Hurray! And of course, the first slew of reader comments, and reviews start coming in. They are all pointing out the typo on page 5. How did anyone miss that?

It’s really no different than the day the software you’ve been working on for a year or two goes live. You’ve run millions of unit tests. You’ve demo it, you’ve made it as easy to use as possible. And 2 hours after it goes out, someone finds a bug that should have been caught 6 months ago.

I think the biggest difference between creating software and creating stories is that with a story, more often than not, you have something to hold in your hands, the product of your long hours of labor. It might be a printed manuscript. It might be a copy of the magazine the story appeared in. It might be a book. It always gratifying, on those rare occasions when it’s happened to me, when someone hands me one of these things and asks for an autograph.

With software, there’s nothing to hold, nothing you can grasp in your hands that represent all the blood, sweat and tears that went into its creation. It’s probably for the best. In thirty years of making software, no one, not a single person, has ever asked me for an autograph. If they did, I’d be a loss: there’s nothing on which to sign my name.


  1. Which explains why this post is coming out at nearly 7pm instead of much earlier in the day

What is a Project Manager?

Finally, I have come across what I consider to be the best definition of a project manager that I have ever seen. I have written in the past about how I dread getting asked that question, “What do you do?” because (a) it is hard to describe what a project manager does without (b) making it sound like a made-up job.

Reading Ed Catmull’s book, Creativity, Inc., this morning, I came across what I consider the best definition of a project manager—one that describes what I do clearly and accurately—but cast in terms of Hollywood production managers. Catmull writes:

Production managers are the people who keep track of the endless details that ensure that a movie is delivered on time and on budget. They monitor the overall progress of the crew; they keep track of the thousands of shots; they evaluate how resources are being used; they persuade and cajole and nudge and say no when necessary. In other words, they do something essential for a company whose success relies on hitting deadlines and staying on budget. They manage people and safeguard processes.

By changing a few words here and there, I have the definition of project manager that I have been seeking for years now:

Project managers are people who keep track of the endless details that ensure software is delivered on time and on budget. They monitor the overall progress of the developmentteam; they keep track of the thousands of lines of code; they evaluate how resources are being used; they persuade and cajole and nudge and say no when necessary. In other words, they do something essential for a company whose success relies on hitting deadlines and staying on budget. They manage people and safeguard processes.

I love this definition. It perfectly describes what I do day-in and day-out on my job. I am particularly tickled by the line, “they persuade and cajole and nudge and say no when necessary.” A project manager who taught me a lot about the job two decades ago summarize this line back then with a simple phrase that I often repeat: “As a project manager, all you have is your charm.”

I’m only a quarter of the way into Catmull’s book, but it has proven its worth with this definition alone. I feel a great sense of relief in having a good, accurate, and succinct way of describing what I do.

Initial Thoughts on the Mac Mini

The new Mac Mini has been up and running for 10 days now and I have some initial thoughts. For context and clarity, I bought the newest Mac Mini (M1 2020), which is running the Apple M1 chip. I bought it with 16 GB of memory (way up from the 4 GB I have on my MacBook Air). The internal disk has 250 GB, but I’ve got two external disks connected to the machine, each of which is 3 TB giving me a total of 6.25 TB of disk space. One of the external disks is for media files and archived data; the other is a local Time Machine backup disk.

As far as performance goes, this machine flies. Applications open so much faster than on my MacBook Air. There doesn’t seem to be any performance hit with backups running and with the various services I have in the background. I really like how fast the machine is.

There are a few downsides I’ve discovered, however.

The M1 chip is the biggest blocker so far. While it is super-fast, not every app has caught up yet, and several still expect an Intel processor. For instance, I use Docker for development work, and I have to run a preview version of Docker Desktop because there is not yet a production version compatible with the M1 chip.

There are some quirks with homebrew as well. Homebrew can be run natively or using Rosetta2 which makes apps compatible, but at a performance cost. Running homebrew natively takes a couple of extra steps to setup, and some bottles have to be built locally to allow them to run natively.

MySQL runs fine on the Mac Mini, but there is not yet a compatible Docker image for MySQL for the M1 chip.

These are relatively minor issues, which only apply to someone doing development work. It appears that most places are working toward making their apps natively compatible with the M1 so I suspect most of these issues will go away with time.

For other tasks: writing, photos, general productivity, I am very pleased with the Mac Mini thus far. Given that it cost significantly less than the newest MacBook Air, it is well worth the cost so far.

I have set up the machine as a home server. I’ve got an internal web server that I am using to build a custom reading list app (that I plan on moving to my domain eventually). I am also using it to host an app for home document archive. Screen-sharing works well with it (I can use screen sharing from my MacBook Air to do development work on the Mac Mini when I am not in my office). I’ll have more to say on these things in a future post.

You can see the new computer in the photo above, peeking between the monitor and the external disks. At some point, I need to clean up all of the cables.

At this point, with the exception of a few development quirks related to the M1 chip, I am very pleased with the new Mac Mini.

Progress on the book collection database

I did manage to get started tonight, but it was mostly on configuration and design issues:

I upgraded my system to PHP 5.1.4 and MySQL 5.0.24. I also upgraded my installation of MediaWiki to 1.7.x (I use MediaWiki for not only developer notes and design documents, but for just about all household related stuff) Once all of that was configured, I got to work outlining the stored procedure API, which now consists of 96 stored procedures.

I expect to do more this weekend, but I’m beat tonight, and am off to bed…

Changes coming to my book database

Ever since I switched from a PC to a Mac at home, I have been using Booxter to manage my collection of books. Back in the day, I had developed my own, Windows-based, home grown application to manage it, but on the Mac, Booxter seemed the way to go. It served me well for quite some time.

But I want something web-based, something that will do libaray and reference management, and something that ties seemlessly into my reading list application. Over the last week, I have been working on the data model in my spare time, and I now have a model that I think does everything I need it to do. To make this work on the web, I’ve decided to do it as a LAMP application (replacing the L for Linux with a D for Darwin): Darwin (Mac), Apache, MySQL, PHP. I’m using a three-tier model for the application design and I have outlined the core API (stored procedures) for the database layer.

I plan on getting started this Labor Day weekend. My goals for the weekend are as follows:

  1. Implement the data model in MySQL
  2. Code the API
  3. Export my data from Booxter into my new data model

If I can get that much done over the long weekend, I will be very happy.

As for the longer-term plan, I am trying to completely seperate the presentation layer from the rest of the application so that the initial interfaces will be bare-bones HTML. Ultimately, I will build up the look and feel using CSS. Once I have a working data management interface, I will look into porting portions of the application to my public website, so that people can search the database if they want to.

Sounds like the perfect thing to be working on during this long and rainy weekend.

I’m also looking into buying two more bookshelves for my home office since I am way out of space on the 7 existing shelve units. But more on this later.

Microsoft sucks!

This is no revelation. But we M$ products at work, and one which I use quite a bit is Microsoft Visual Studio 2003 and Visual Source Safe. This morning, after compiling an application successfully, Visual Studio went bye-bye silently. No application exception, nothing in the event log. Nada. A reboot didn’t seem to fix the problem. I’ve narrowed down the problem to projects that are checked into Visual Source Safe. I can’t seem to open those projects, but projects that are not checked into revision control are accessible.

And yes, I checked the VSS server and I can get into that just fine.

This sucks becuase I was about to finish something up and get it published and check it off my list and move on to the next task. Now I’ve got to spend time I don’t have trying to figure out what the heck is going on!


UPDATE: It’s all working now, although it took a threat of setting my computer out to sea and praying for a hurricane to get it to cooperate. Incidentally, after I clicked the “Save Post” button on the initial post (when, in fact, I felt like screaming!), Tears For Fears, “Shout” came onto my iPod:

Shout, shout,
Let it all out.
These are the things
I can do without
Come on; I’m talking to you
Come on.

What a coincidence, eh?