Since I am on vacation and happened to find myself with an empty hour this afternoon, I managed to clean up my code enough to where I was willing to put my Obsidian daily notes automation script on GitHub. This is the script that I use to automate the creation of my daily notes in Obsidian.
I’m posting this software as-is. It works for me, and a number of folks have requested it and I’m happy to put it here to share it. But I have no time to support it. If I make improvements, I’ll try to post them, but there’s no guarantee there either. I realize that this may not work perfectly on non-Mac systems, but the whole point of posting the code is to let folks see it, fork it, and roll your own from it. Hopefully it works for you the first time. If not, the code’s there for you to mess with.
For those who choose to use it, keep in mind that I run my script on a Mac, and icalBuddy, which I use for pulling in my agenda, is designed for Apple’s calendar app. You may need to look for alternatives for other platforms.
I’m always eager for feedback and suggestions, but as I said, my schedule is such that I don’t have time to provide any kind of support for getting the script working for you.
While reading Walter Isaacson’s new book, The Code Breaker, I was particularly struck by some seemingly minor details. The book is a fascinating look into the modern process of scientific discovery, and there was some discussion of how a discovery written in a lab book and then signed by witnesses in order to document the dates of the discovery. When do scientists learn to do this?
I took AP biology, and AP physics in high school, as well as physics, chemistry and organic chemistry in college and no one every taught me how to properly use a lab book. Indeed, what was implied, at least at that level, was that what the teaching assistants and grad students who led the labs really wanted was nice, neat copy in our lab books with clear results that were easy to grade. I remember many of my fellow students had two lab books: the one they worked stuff out in, and the one they turned in after everything was cleaned up. I couldn’t spend the money on two lab books, so mine were messy.
It seems to me that the mechanics of a lab book–its true purpose and how it is used the real world–is a practical lesson that any burgeoning scientist should learn. But who teaches this? Are there upper division chemistry classes that focus on this? Certainly o-chem didn’t.
This got me thinking about other practical lessons that I would have benefited from, but was never formally taught. How to read a newspaper is one example that I’ve written about before. What about keeping a diary or journal? I don’t ever remember this being taught in school. I don’t ever remember a class in which the pros and cons of journals were discussed. I would have found these things very useful. Instead, I learned how to keep a journal by following (initially) the example Isaac Asimov described for himself in his autobiography.
Lab books are useful tools outside of the laboratory. For the first half of my career, I didn’t keep any kind of notes about the code I was writing. If I had to recreate something, therefore, it was often hard work. At some point, it occurred to me to keep notes as I worked. When I do something particularly complicated, I often list it out in my notes in high level steps, and then fill in the details as I work. I keep one simple idea in mind: a person new to the organization should be able to take my notes and reproduce my work. Technical debt is a big problem in I.T. People come and go and leave behind lots of undocumented code in their wake. You’d think lessons in keeping good notes would be part of the training process, but I’ve never seen it.
For that matter, how about something as simple as keeping a to-do list? I was never taught this in any of my classes.
There was one class I had–a 7th grade science class–in which our teacher spent quite a bit of time teaching us how to organize our work. We learned how to keep our science folder, and how to keep our notes and assignments organized in the folder. It was practical information that served me well through the rest of my pre-college schooling. Beyond that, most of the practical things I learned from books.
I can’t remember a teacher teaching how to take notes: how to identify the important points, and highlight them; what to leave in and what to exclude from the notes; tricks of shorthand to capture information more succinctly. All of this I had to figure out on my own. I read a book between my sophomore and junior years in college, and one chapter was all about note-taking. It changed the way I take notes and I use that method to this day.
I try to pass on some of these practical lessons to my kids. The Little Miss keeps a journal and I encourage that, and allow her to look at my journals in order to take ideas, but mainly so that she understands she can make it whatever she wants it to be. The Little Man could benefit from a daily to-do list, and I’ve tried on a couple of occasions to suggest it, even offering to help him get started by reviewing it together. He resists it, but he is at the age where he doesn’t think he needs it. (He does.)
It seems to me that in addition to classes in science and math and reading and English and history and art and physical education, there should be some practical classes on topics like these. Better yet, practical lessons could be merged into the existing classes.
In science, you could learn how to keep a lab book while you do your experiments. The lessons would be about the purpose–not to show you got the right answer, but to be able to reproduce your results, whatever they were.
In English, there could be a section on the literature of diaries and journals. There are plenty to choose from: John Adams, Samuel Pepys, Henry David Thoreau, Anne Frank just to name a few. Discussions could ensue about why to keep a journal, the practical value, and the literature can provide examples of what other people have done.
In home room, you might learn how to better organize your day, keep track of your work, and manage stress.
We need more practical lessons. I certainly would have benefited from them earlier than I did.
Author’s Note: This essay is the second in a series on how I learned to write code. You may want to check out Part 1 if you haven’t already done so.
By the time I got my VIC-20, the computer was already outdated, but I wasn’t complaining. There was only one problem: I didn’t know how to write code. At least, not beyond the few commands I’d learned the previous summer. Moreover, being in a new town, I didn’t know anyone who did know how to program a computer. I was on my own.
What I did, then, was get my hands on some computer magazines. In the 1980s, computer magazines were popular on the magazine shelves, even in the grocery stores. I ignore most stuff in the magazine and focused on the programming samples that they had. I have no memory of what any of these programs did, but I remember some of them spanned multiple pages. I would carefully type these programs into my VIC-20, line numbers and all, and then run them. Often, they’d generate an error as soon as I executed them. Usually, this was because of a typo I’d made. It might seem like an inconsequential thing, but I began to develop a lightning focus for these typos. Today, when I compile something, or run some code I’ve written, and it errors out, I can almost instantly see the problem (forgot a semi-colon at the end of a line; or, a bad indent on a Python script.)
When the programs worked, I was delighted. I also began to experiment. I would change lines of code to see how it changes the program, and slowly I began to deviate from what was on those pages to something different.
This was slow going, made slower by the fact that it took a long time to enter these programs, and the tape drive I had didn’t always save them reliably. If that happened, and I wanted to run the program again, I had to start from scratch. On the downside, this meant a lot of re-entry of code. On the upside, I gained a lot of experience and got more familiar with the code I was entering.
Still, it seemed a slow way to learn. I needed something faster.
Eventually, my VIC-20 was replaced with an Apple IIe, which in turn was replaced by an IBM PC. It was on these two computers that things began to speed up for me. For one thing, both computers used floppy disks instead of tape drives, which made saving the code I typed in much easier. For another thing, I discovered a more practical way to learn to code: by having a specific, concrete goal in mind.
It was on these computers that I first encountered text adventure games. There was the Zork series of game, of course, which had a multiword parser that was almost like typing English sentences. But what grabbed my attention more were the Scott Adams adventure games, which used a simpler 2-word parser. Until i discovered these adventure games, I didn’t even know what the word “parse” meant. It hadn’t come up in any of my English classes. And as it turns out, parsing, is a fundamental operation in both operating systems and computer languages. I didn’t know it at the time, but these adventure games were helping me learn some basics of computer science.
Scott Adams adventure games were like Choose-Your-Own-Adventures, but more elaborate. You were presented with brief descriptions and then presented with a prompt like:
You are standing in a dark room. There is a door on the north side of the room and a hallway to the east. A letter rest on a desk in front of you.
There was a prompt and you could type in two word commands, which would evoke different responses. For instance, you could type:
which might produce a response: “You don’t have a letter to read.” So you’d type:
and see: “You pick up the letter from the desk.” Then:
and see: “Do not open the door. It is booby-trapped.”
The games interested me as a unique form of story-telling. (I was just beginning to write my own stories around this time.) But I was intrigued and fascinated by the mechanics of the game, even more than the storytelling. I tried to imagine what the game looked like inside, like opening up a clock and peering into the elaborate gear-works within. Only, I knew this wasn’t just gears. This code, and I was becoming more familiar with code. It occurred to me that this could be my goal: to learn how to make my own adventure game by learning how these games worked inside.
Right off the bat there were three things about the game that interested me:
How the parser worked.
How the map for the game worked.
How the game knew when things changed within it.
Much later, I realized that through these three interests, I was learning three fundamental concepts about programming:
Parsing and interpreting data.
Storing data and relating data in useful ways.
Managing the “state” of a system.
In almost every piece of software I have created in my professional life, each of these three concepts place a significant role.
I had to really look around to figure this stuff out. There was no Internet at the time. There were bulletin boards, but I didn’t have a modem to connect to any of them, so I was mostly on my own. The computer magazines helped some. Do did the Granada Hills Public Library, where I could go and look at books on computer programming.
I remember I figured out how to write a 2-word parser entirely on my own, in much the way I imagine a person who can’t read music learns to “play by ear.” There was a lot of trial and error, but eventually, I managed to be able to split 2 word input into a “verb” and “noun”. I could compare those to lists and if the entries weren’t in the lists, I could return error messages.
Figuring out the map was much more difficult. Ultimately, I found a way to create a map using BASIC “data” statements. “Data” statements was how data that a program used internally was stored a part of a BASIC program. Imagine an adventure game with 4 rooms. There are 4 cardinal directions that are possible from each room. Suppose you have a map that looks as follows:
The data statement for a map might look something like this:
DATA 1, 'You are standing in room with a door to the north and a hall to the east', 3, 2, 0, 0
DATA 2, 'You have entered a hallway. There is a room to the west.', 0, 0, 0, 1
DATA 3, 'You are in a cave. A door leads south and there is a sldie to the east.', 0, 4, 1, 0
DATA 4, 'You are stuck in a pit at the bottom of a slide.', 0, 0, 0, 0
The first number is the room number as shown on the map. The next item in the statement is a description of the room that is displayed when you enter it. The following four numbers represent the rooms you can get to for each of the cardinal directions, north, east, south, west. So from room number 1, you can get to room number 3 by going north, and room number 2 by going east.
Figuring out the “state” of the game was probably the most difficult. Consider the letter on the desk. If I tried to read the letter without picking it up first, the game would tell me that I didn’t have a letter. But if I picked up the letter, I could read it. The game had to know that I had picked it up before I could read it. Once I picked up the letter, the “state” of the game had changed. But a game could have dozens or hundreds of situations like that. Door that were closed or opened. Lights that were off or on. Puzzles in these text adventures were largely based on the state of the game.
Eventually, I figured this out, too, although it took a little help. For the first, and only time, I enrolled in a class on computer game programming that was being given at California State University Northridge. I think it was a Saturday class through their extension program. I do remember that I was the only “kid” in the class. That class was helpful in that it provided a useful structure for how to think of things like “state”. It also taught me how to create a game where the “monsters” in the game were taught to find you, regardless of your position in the game map. (Much later, I realized this was an algorithm, but I didn’t know it then.)
I kept at it. I created my own adventure games which, in their look and feel, seemed indistinguishable from Scott Adams adventure games. It was only in their story-telling ability that they flagged. I tried to make my games too complex, with maps with a hundred rooms instead of a few dozen. I was less interested in the storytelling and more interested in the mechanics. The more code I wrote the better I got, but it was almost always off-the-cuff. They were also always plain text games. I felt I needed to expand my wings a bit.
I began looking for practical applications for my programming, instead of just building something to see if I could. One thing I coveted was Microsoft Flight Simulator, mainly because I’d always wanted to be a pilot, and flight simulator was about as close as I could get as a teenager.
The opportunity came as part of a project in a computer programming class I took in either eighth or ninth grade. Don McLish was our computer teacher and our project was to built something using the concepts we had learned. I built a flight simulator. It was a fairly simple thing. I think there was a single instrument, and line that represented the “horizon” out the window. But it moved the way it should as the plane steered through the air. I remember it requiring programming using some trigonometry, which I was taking in math at the time. It impressed the teacher enough for him to sent a note home to my folks.
This encouraged me, and despite the occasional comments that I was “spending a lot of time” on the computer, I continued to practice writing code, more and more looking for practical applications. In high school, worked in a local pharmacy and was impressed by the software the pharmacist used to manage his customer database. I went home and tried to mock-up my own version of that software as a kind of personal “rolodex” program on the IBM PC that I had. It worked so well that I used that program for several years, including my first few years in college.
In college, things slowed down a bit as I was focused on classes and my interactions with computers was mostly for typing up notes and papers. But two things took place in college which would have a big impact on me and my programming.
The first was a computer class that I took with one of my roommates. I think we learned to program in Pascal, a teaching language. That class made me realize that most computer languages were more alike than they were different. It wasn’t a big deal for me to go from BASIC to Pascal once you knew the subtle differences. We also used NeXT workstations our labs for that class, and I was impressed by the NeXT, which was Unix-based underneath. It was a fleeting thing, because the class was just one quarter, but I didn’t forget the NeXT or that initial introduction to Unix.
The other thing that took place was a promotion I got. I’d been working in the dorm cafeterias for my entire time in college. I’d started in the dishroom (which I enjoyed) and worked my way up to a supervisor, and then, in my senior year, into the business office, where I ended up spending the year writing computer programs to automate the cafeteria budgeting process in Microsoft Excel. I was now being paid to write computer code, and as much as I enjoyed working in the dishroom, I enjoyed this work a lot more.
Recently, I passed a personal milestone. It is an arbitrary milestone, one for which I am the sole judge, but it is one that has been 37 years in the making. I have been a professional developer (coder) for about 27 years now. In that time, I’ve generally had no problem considering myself a professional. No impostor syndrome there. But there is one thing from which I have refrained referring to myself: an expert.
Until now, that is. Now, after 37 years of learning how to write code, I think I do it at a level which could be considered expert. Perhaps even by someone other than myself.
“Developer” sounds a little phony to me. I’ve read about people who are developers (land developers) and I’ve never really been clear on what that entails. I prefer referring to myself as a (professional–now expert) “coder.” It’s less formal, but despite its reputation, I’ve always felt there is something informal about coding. I’ve been wary of the term “expert.” It seems to me that it is overused to such an extent as to water down the meaning. I’ve seen all kinds of books about becoming an “expert” this or that in 30 days. Maybe I’m just bitter at being a slow-learner but it took me 37 years before I finally considered myself an “expert.”
One thing I don’t consider myself is a software engineer. An engineer has a specific meaning in my mind, and entails a certain kind of formal education in software development that I lack. I am not an engineer. I am immediately suspect when I see “software engineer” on a resume when I don’t see an engineering degree along with it. I’m much less suspect if I see “coder.”
I tend to be a slow-learner, perhaps because I dive in head-first and try everything at once. I’ve written about how it took me 14 years of writing and submitting stories, collecting more than a hundred rejection slips before selling my first story. It took me 11 years of teaching myself to write code, before I landed my first (and so far, only) professional gig. In that regard, it only took another quarter century or so before I felt I could call myself an expert.
So how does one learn to write code in 37 short years? For me, it began with hangman, and WarGames.
The first computer I ever saw was a Commodore VIC-20. I saw it in my 5th grade math class sometime in the late winter or early spring of 1983. There are exactly 3 things that I remember from that math class. First, our teacher was missing part of a finger. Second, one of our lessons was learning to read the stock pages in The Providence Journal. Third, was the Commodore VIC-20.
There was no math associated with our introduction to that computer. I remember it was wheeled into the classroom, connected to a television set. We spent the class using the VIC-20 to play Hangman. I didn’t see a line of code during that introduction, but I was intrigued by what I saw.
The summer of 1983 was my last on the east coast before I moved with my family to Los Angeles. It was the summer that WarGames with Matthew Broderick and Ally Sheedy made its debut. I saw the movie in New York with my cousins. I don’t recall the movie making much of an impression one way or another at the time. What I remember most about that day was going back to my cousin’s house after the movie, and being introduced to his Timex Sinclair 1000. It was the first computer I ever laid my own hands on.
My cousin turned on the computer and showed my how to write a simple program in BASIC. The program was:
10 PRINT "HELLO"
20 GOTO 10
The program doesn’t do much, but something in my brain clicked. It was like I understood the concept of programming in that instant. With a finite (even small) set of instructions, and some basic logic, you could make the computer do all kinds of things.
That evening, we made the computer break into a top secret installation. We didn’t have a modem or any kind of connection to the outside world. But using our memories of WarGames, and my quick absorption of BASIC, we wrote a program that made it seem like we were hacking into some secret computer system. I can’t remember what I program looked like, but it was probably something like this:
10 PRINT "WELCOME TO TOP SECRET INSTALLATION"
20 PRINT "Enter your password:"
30 INPUT x
40 IF x = "password" THEN GOTO 50 ELSE GOTO 60
50 PRINT "Welcome to Global Thermonuclear War"
60 PRINT "Wrong password"
Yeah, we didn’t have the logic quite right, but the idea that through some simple instructions you could make the computer to all kinds of things was a revelation for me.
At the end of that summer, we said goodbye to the east coast, and hello to Los Angeles. Any move is tough on an 11-year old, but moving across the country, away from all of your friends is particularly tough. But I had something cooking in my mind that I was looking forward to. I was going to figure out a way to get my own computer. I didn’t particularly like the keyboad on the Timex computer. What I had in mind was the VIC-20 I’d seen in my 5th grade math class. I thought about it so often that I’d dream about it. I remember a couple of occasions that I dreamed I’d gotten a VIC-20. I was so excited! Then I’d wake up, uncertain at first if it had been a dream, and then, crestfallen, that it had.
Until one day, I had one! My very own Commodore VIC-20. And it came with a tape drive! My real coding experience was about to begin…
I spent the entire day working on SQL stored procedures for importing data. Pretty routine work, but I did some innovative validation stuff, that should make it easier for a person to validate the data that gets imported into the database. I was in good flow and both the morning and the afternoon flew by.
I’m home now and setting my sights on getting my next assignment done for the writing workshop. Week three is on setting so I’m sitting the comfy chair in the family room, surrounded by dining cats, with my laptop and workshop book. Earlier today I had some insight into how the new version of the story will unfold and I think that will help with the writing of the next scene. But I’ve got some chapters to read first.
No mail when I checked a little while ago, but that may be because it hasn’t been delivered yet.
Nearly 130 pages through Vinge’s Rainbow’s End, and I’m finding myself both amused and entertained by it. It was tough not to use this hour or so before Kelly calls to just read more, rather than do workshop stuff.
Kelly and I planned for these to be a peaceful weekend–our first one in over a month, since moving to the new house. And for the most part, today was very peaceful. I went to bed early last night–by accident. I got in bed to read, and before I knew it, I just couldn’t focus on the page any longer. Kelly came to bed later. I managed to get 10-1/2 hours of sleep, which is remarkable for me.
We were still up before 8 AM. We lazed around for the first two hours, watching some TV. Kelly Tivo’d "Rock the Reception" on TLC, which was a lot of fun to watch, and really kind of addicting. We set the DVR to record it some more whenever it comes on. I worked on some wedding website stuff for a while. Finally, just after 10 AM, we headed out for breakfast. We walked a different route to Shirlington, through Park Fairfax. It’s a slightly longer walk but I like it much better. We headed to the Luna Grill for breakfast, where Kelly had eggs Florentine, and I had a bagel and lox. Then we walked back home.
We lazed around some more. Later in the afternoon, Kelly and I put together the new "kitty bathroom" that she ordered. I did a little reading and then we headed out to Pentagon City to do some shopping. We got some groceries for the dinner I was planning to prepare. We also got some bike stuff. We got a new tube for Kelly’s rear tire, and I got a new helmet and a mirror.
Back home and I started to prepare dinner. I planned on making Chicken Marsala with a side of Italian Pasta Salad. I prepared the pasta salad first, making the dressing, chopping up lots of veggies, and then refrigerating the whole thing. Then I started on the chicken. We used our new dishes and napkins and sat down to eat right around 7 PM. And guess what? For a wonder, it came out really good! The chicken wasn’t overcooked, the Marsala sauce was just right, and the pasta salad was great! This time, Icooked the chicken on a lower flame, but for a longer time than the cookbook said. I was pleasantly surprised by the whole thing!
After dinner, we booked the final excursion for our honeymoon cruise (this one, at Ocho Rios, Jamaica, takes us to some caverns and then some waterfalls), and then headed off to the gym. I got in a decent lower body workout.
Got the most recent issue of SCIENTIFIC AMERICAN in the mail today. Actually, it was delivered to a neighbor, but they kindly brought it by. It’s a special issue on privacy and looks like a good one. I’m somewhere around 220 pages through Old Man’s War, and expect to finish tonight or tomorrow.
Almost no one who reads this is a programmer of any kind, so this will mean very little to you. But I just have to say that I spent the last 2-1/2 hours trying to debug a type problem in a C# .NET application I am working on. I added a cool feature that allowed a table of dates to be edited inline from the browser, but when the date was submitted to the database, I kept getting an exception indicating that the string could not be converted to a datetime. I tried different approaches. I made little tweaks. I scoured the InterWeb. Nothing was working and yet I was confident that the value I was passing was a datetime value.
I was about to give up and go to lunch when a stray thought struck me. Maybe the value is a datetime, but maybe the parameter which I am setting using the value is not a datetime. I checked, and sure enough that was it. Someone, in the hours of frustration, I had swapped the order of two lines of code. That was all that was wrong and all of my frustration and fruitless attempts were for naught, if only I had recognized this change 2 hours ago.
There were some problems with the development server that I was using this morning. What made it worse is that the server is in Santa Monica and I can’t just walk up to it on smack it. I couldn’t even make a remote connection to it. Aside from other things, this server is our source control server for all of our source code and I was unable to check out some code I needed.
Fortunately, some folks were in the Santa Monica office this morning and were able to bounce the server and I am finally (after over an hour’s delay) able to get some productive work done.
Well, what I thought was going to take me a full eight hours took under four. It must be the music! I can now move onto some other stuff–in particular a bug that was reported late last week in some software that is rolling out on March 5.
Today is a code-writing day at work. There is also construction going on outside the building and there is intermittent bone-jarring noise that has already developed into a headache. Headaches are rare for me so you get the idea. I was about to put on my iPod to block out the sound when it occurred to me that I have been meaning to post what are, for me, the top 10 albums for writing code. Keep in mind that I personally feel more productive when I listening to these albums while casting out spells of C# or Transact-SQL or PHP or Perl. The downside is that I am so focused, I usually don’t hear the songs. The list is alphabetcial by album title.
I went to happy hour after work last night with several people from work. We went to Chammps and we were there until just about 8 PM. I was home just before 9 PM and I tried to read for a little while but I was pretty tired and I went to bed soon after.
I was up at about 8:30 this morning and headed up to IHOP in College Park for a big breakfast. The place was packed. A tour bus had arrived just before I got there and it was chaos. Fortunately, because I was a party of one, I got seated quickly, and had my scrambled eggs, bacon, hash browns, and pancakes post haste.
When I got back home, I worked on the book collection database for a few hours and did a little laundry. I’m eating lunch now and plan to head over to the gym for my arm and shoulder workout at about 2 PM or so. I’m in desperate need of a haircut so I may try to get my hair cut after the gym.
I plan on getting some writing done today at some point, but I’m not sure which story I am going to work on yet. I think I need a little more planning on both and that will give me a chance to test some of the features of Scrivner.
Last night my car ignition was getting stuck when I put the key in. It wouldn’t turn. This has happened before and I think it is a known problem with older Saturn’s but last night was the first time it happened in a while and it took some jiggling before I could get the key to turn. I’ve been careful about it since then and so far so good. But it’s another thing to keep an eye on.
I spent about 6-7 hours today working on interfaces for my book collection/reading list databases. It’s possible that by tomorrow I will have enough in place to make some of this more public so that people who are curious can take a look.
I’m quitting for the night on this, and I’m just going to relax for the rest of the evening.
The driver side door on my car has a problem. When I opened it the other morning, I heard a pop. The cable that attaches from the car to the door to help the door open slowly and keep it in place snapped. I vaguely recall this happened to me once before many years ago (or maybe it was Tawnya’s car?). It’s not a problem, I just have to be careful when I open the door that it doesn’t fly open. But at some point, I should get it fixed.
I got my T-Mobile bill today and sure enough, I completely forgot to pay it last month; the new bill was exactly twice as much as usual. Oops.
A few minutes ago, I received email from Norm (of vickyandnorm fame). He emailed me from his flight from London to New York asking if it was unusual that the outside temperature at 38,000 feet was -75 C. For some reason, I found that amusing.
In the course of the last 3 hours that I have been writing code, I have listened to 46 songs in iTunes “party shuffle” mode. That’s an average of just under 4 minutes per song. I don’t know what made me think of checking that, but I think it means it’s time to go.