Category Archives: personal automation

How Much Time I Spend Writing, Automated and Revisited

About a month ago, I automated the process of capturing how much time I spend writing each day, and incorporated that data into my Google Doc Writing Scripts. Here is how this work:

  • I use RescueTime on all of my computers, home and work.
  • RescueTime tracks how much time I spend in various applications, including specific documents.
  • Using the RescueTime API, I wrote a script that captures how much time I spend in Google Docs each day.
  • That number gets recorded in my writing spreadsheet automatically each night.

This means I no longer have to “clock in” or “clock out” to track my writing time. I just start writing, stop writing, continue later, etc. and all of it captured automatically by my script. With almost a month of this type of data on the books, it’s interesting to look at how my guesses match reality.

Generally, when I’m firing on all cylinders, I can write 6 pages (1,500 word) per hour. Put another way that is about a page every ten minutes. Of course, I don’t always reach this apogee of output. It turns out (with about 30 days worth of data to go on) that the correlation between the time I spend writing and how much I write is pretty strong (0.59). I took the data and ran a scatter plot, with a trendline using that correlation, and here is the results:

Writing Time

It is clear that the more time I spend, the more I write, but it’s not as strong a correlation as you might think. Part of the reason is that sometimes, it takes a while to get things out of my head. Here is what that same set of data looks like plotted individually over time. First the word counts…

30 Days Words

and then the time spent…

30 Days Time

These two charts illustrate that while the correlation is pretty strong, there are times when I clearly get bogged down. August 5 is a good example. I wrote just about 1,200 words, but it took me 79 minutes. And yet on August 7, I wrote 1,600 words and it took me under an hour. This variability is caused by two things:

  1. Concentration. Sometimes, in difficult scenes, I slow way down to think things through and work them out. Remember, I generally don’t plot ahead, so especially in first draft, I’m working out things on the fly.
  2. Interruptions. I’ve talked about how in order to write every day, I’ve had to learn to write with distraction. Sometimes, the kids will need me for something, I’ll step away for 5 or 10 minutes with no progress on the document, and then return and write more. That clearly shows up as slower.

But that red trendline in the first chart is pretty accurate, and comes close to my intuitive guesses. I have said that I wrote about 500 words in 20 minutes. That’s 1,000 words in 40 minutes. If you look at the 1,000 word-mark on that first chart, and then go up to where the red trendline crosses the 1,000 word-mark, it’s right about the 40-45 minute mark. My intuition is pretty accurate! You’ll also note that 1,500 words crosses at right about the 60 minute mark.

I have less than 30 days of the time data, but as this volume of data increases, I expect the trendline to become more accurate. One thing that is particularly useful about a chart like this is that it can tell you for a given amount of time you have available, how much you can accomplish. Or, flipping it around, if you want to write 1,000 words, how much time will you need to set aside?

Entirely automated

I wanted to call this out one more time. All of the data above is generated automatically. I don’t spend a single instant of my time collecting it. That is perhaps the biggest value. Once I wrote the scripts (which I did spent time on) I get the data without any effort, and this can be used to help me make adjustments down the road.

You can see my realtime data, including how much writing I’ve done at various intervals (my ongoing writing streak, for example) and how much time I’ve spent writing. Head on over to open.jamierubin.net to check it out.

The Best Way to Contact Me

Last summer I retired my voicemail. Voicemail is an antiquated communication mechanism and does not lend itself toward automation or speedy responses. Recently, I’ve noticed an uptick in the number of calls I get from phone numbers that I don’t recognize. About 90% of the time, when I answer these calls, they are solicitations of one kind or another. I’m tired of them. I don’t mind saying “no” to these solicitations, but it does grate on me when they don’t take no for an answer.

So, going forward, I’m not answering calls from numbers that I don’t recognize. That means that going forward, the best way to contact me, especially if I don’t know, is though email. The email address to use is:

jamie [at] jamietoddrubin [dot] com

In order to encourage this as my preferred form of contact, I am going to attempt to answer most messages within 24 hours.  To keep me honest, I’m making my real-time email response stats available for folks to look at1  As of July, those numbers look as follows:

July 2014 Response

You can see that right now, I answer about 30% of my email within 24 hours. Over the next month or two, I will try to bring this up above 75-80%.

When in doubt, send me an email. It is always the best bet for the quickest response.

Social media

I’m usually pretty responsive on social media, almost always responding within the same day, although the time within the day can vary depending on how busy I am. Feel free to reach out to me on Twitter, Facebook, or Google Plus for things that don’t require a private email message.

Blog comments

I really try to stay on top of blog comments, but I also like the thread to stay on topic. For things that aren’t related to the post at hand, you are better off sending me email, or getting in touch with me on social media.

Chats, Hangouts, Skype, etc.

I’m perfectly fine doing Google Chats, Facebook Chats, Google Hangouts or Skype, but again, these depend on my ability to be fully engaged. If the question or request is fairly simple, email is almost always your best bet.


Of course, if your number is in my contacts, or we’ve pre-arranged a phone call, we’re good. I’m just drawing the line on unrecognized numbers because they turn out to be a steady drain on my very limited time.

Have questions? Drop them in the comments and I’ll do my best to answer them.

Notes

  1. Not quite real-time yet, but they will be in they will be within the next few days. Right now, it is displaying my static aggregate numbers from July 2014.

Introducing open.jamierubin.net

With all of the data I collect about myself, I’ve been wanting to put together a kind of open dashboard that provides a window into the data through interesting visualizations. While my short term plans are nothing like the amazing things happening over at Aprilzero, I have put a very early prototype together of what I am calling open.jamierubin.net.

open.jamierubin.net

Right now, all the site does is make a live query to my Google Doc Writing Tracker spreadsheet, and renders the data in a chart on the site. Clicking on the links above the chart, you can see either the last 30 days of my writing data plotted out, or go back to the beginning of time (over 500 days).

For each visualization I publish, I plan to include a link to the “HOWTO” which will include the code I used and how I pulled the data I needed to make the visualization. That way, if others want to give it a try, there will be at least some documentation.

Eventually, I will come up with a framework for the site, and begin pulling in other data as well. For now, this is a quick-and-dirty prototype of what is possible with just a little bit of code. Take a peek at it and let me know what you think.

Open Beta of My Google Docs Writing Tracker Version 2

I have done a major refactoring of my Google Doc Writing Tracker. Several new features have been added, but the biggest change is that the scripts are not data-driven, making them much easier to setup and configure. If you are using the scripts today, or have been wanting to try them out, you are welcome to install the new beta version.

You can get the files from the beta-version-2 branch on GitHub.

Be sure to read the README as that has been updated to reflect the changes in the system.

New features

  • NEW: A new spreadsheet is available with all of the configuration information built into it. You just fill in the blanks on the Config tab, and the script takes care of the rest. This makes it easier to configure and customize without the need to go into the code.
  • NEW: Option to break down daily word counts into fiction/nonfiction.
  • NEW: Ability to run the scripts in test mode. Allows you to see the results of your configuration in the log without the changes being applied to your spreadsheet.
  • NEW: Ability to customize the order of the columns on the Writing tab.
  • NEW: Ability to customize the names of the tabs in the spreadsheet.
  • NEW: Ability to customize the location and names of the Sandbox and Snapshot (formerly “Earlier”) folders.
  • NEW: Ability to capture time spent writing by integrating with the RescueTime API (experimental).
  • NEW: Ability to generate Daily Almanac summary email that lists stats for the day, and identifies trends and records.
  • NEW: Improved logging in test mode.
  • NEW: Validation of configuration settings.

What I hope to accomplish with the beta

So far, these changes are working very well for me. But I can only really test in my own environment, and because I initially wrote the scripts for me, they may be inadvertently tailored to my environment. In this version, I’ve tried to generalize a lot of the code and make it more flexible and easier to use in other environments.

What I am looking for in this beta is to have people test the scripts in many environments in order to identify any problems, and iron them out before merging this code back into the master branch.

To that end, if you use the scripts I ask that you do 2 things:

  1. Log an issue if you find a problem. Understand that I don’t have a lot of time to work on these scripts. I set aside a chunk of time once a year or so to do a major refactoring like this, but that’s about all I can do. So while I will try to address all of the issues, it may take a while.
  2. If you see a way in which the documentation can be clarified, by all means let me know.

If I do have time, I will try to address the issues as quickly as I can, but that time isn’t guaranteed, and as I say in the README, while I’m making these scripts available to anyone who wants to use them, I don’t have time to support them. You use them at your own risk, so be sure to read the README.

The initial setup can be a little cumbersome and I’ve tried to clarify it in the documentation. Once it is setup, if all goes well, it should just run silently in the background and add to your spreadsheet each day.

If you use the scripts, let me know how they work for you. They work great for me, but of course, they were designed for me and my environment. With this revision, I’m hoping that they work equally well for anyone who chooses to use them.

The Daily Almanac Has Been Added to My Google Writing Tracker

One of the most frequent requests I get regarding my Google Writing Tracker is to make my Daily Almanac available as part of those scripts. The wait is over. Today, I pushed out the Daily Almanac the Google Writing Tracker project on GitHub.

For those who don’t know, my Google Writing Tracker is a set of script that automate the process of tracking what I write every day. Since I do all of my writing in Google Docs, these scripts run automatically each night, look at what I wrote, tally up the stats and record them in a spreadsheet. They also email me a copy of all of my writing for that day, including differences from the previous day.

Along with those scripts, I built another script that I call my Daily Almanac. This script culls that spreadsheet that is populated by my Writing Tracker scripts and gives me a summary report each night. The report tells me how much I wrote that day, and breaks it down for me. It also identifies any streaks I may have set (369 consecutive days of writing as of today) and any records I may have set. (The most words I’ve written in a day, etc.) I set up my Daily Almanac to send the nightly email to Evernote so that I have a nice record there of my day-to-day writing activity. Here is what a typical Daily Almanac entry looks like:

Daily Almanac July 23

The Daily Almanac is now available for anyone who wants to use it with the Google Writing Tracker. I have checked it in to the project on GitHub, and I’ve updated the README file with detailed steps for setting it up.

As always, this is a use-at-your-own-risk thing. I just don’t have the time to support these scripts. The best I can do is make them available for others who want to give them a try, and encourage folks to add to improve upon them. Be sure to read the instructions carefully, and if you do find any bugs, feel free to open up an issue in the GitHub project. I may not fix it any time soon, but at least it will get tracked.

Alpha Testing an Update to My Google Writing Tracker

Beginning today, I am doing some alpha testing of the first significant update to my Google Writing Tracker scripts in more than a year. I will be testing these out myself over a period of a week or two before pushing the changes to GitHub.

The newest feature is that the writing scripts now track both fiction and nonfiction writing. This may not seem like much, but it is a big deal for me, as I have been writing a lot more of the latter lately and want to be able to look at the data to see how much of what I write each day is fiction, and now much is nonfiction. Fiction and nonfiction columns are captured separately in the Writing spreadsheet, and a third column keeps track of the total writing, fiction and nonfiction.

My Daily Almanac has been modified to report on this. Here is what a new version of my Daily Almanac email looks like when it is sent to my Evernote account:

DailyAlmanacNew

The script distinguishes between fiction and nonfiction by looking for a tag I include at the end of my template document:

  • {{Fiction}} = Fiction
  • {{Nonfiction} = Nonfiction

I am also working on a few other changes:

  • I’ve added the ability to run the script in a test mode, that sends the email containing what you wrote that day, but does not actually update any values.
  • I’ve added a check to make sure that the script only looks for Google Doc files in the sandbox.
  • I’m working on simplifying the setup process.

It will probably be 2 weeks before I push these changes to GitHub. However…

I have added my Daily Almanac script to the GitHub project today because I know a lot of people were asking for that. Stay-tuned for the next post for more details.

Reminder: My Google Writing Scripts are Available on GitHub

After my inaugural post for The Daily Beast appeared, I’ve been asked almost daily if the scripts I mentioned in the post are available. They are available on GitHub. I put them there last July. I hesitated to mention them in the post on TDB because I didn’t want to come across as promoting my own stuff. But since I’ve been asked almost daily since the post appeared, I’m thinking that maybe I should have. Ah, well. If folks are interested in trying out the scripts, or improving upon them, you can access the code on GitHub. Be sure to read all of the instructions there to get them working correctly.

All About My To-Do List System

I get asked from time-to-time about what system I use for my to-do list. I use a plain-text, command-line system developed by Gina Trapani called todo.txt. In concept the system is very simple. There is a text file containing all of my todo items, one item per line. There is another text file containing all of the things I’ve done, one item per line. There are commands that I can run to add, complete, and otherwise manipulate my to-do list. Over the years I’ve tried most of the apps and services out there, but I like todo.txt for 4 reasons:

  1. It is surprisingly simple. It’s just text files.
  2. You can do complex things with text files.
  3. Text files are compatible with just about everything.
  4. I can integrate the system with just about anything.

I thought I’d give a walk through of how I use todo.txt to manage my to-do list in order to illustrate some of these points. I’m not going to go into the details on installing the system, as there are already clear instructions for doing this.

What goes on my to-do list

I put 2 kinds of things on my to-do list:

  1. Anything that I need to do that I don’t want to have to remember.
  2. Anything I’ve completed and want to track after the fact.

The second thing may not be obvious, but often times I’ll add something to my to-do list right after I’ve done it, and then immediately mark it as done, so that I have a record of it that I can look up later.

Todo.txt allows you to mark up your list in certain useful ways. A + sign in front of a term represents a project. We are remodeling our kitchen so things related to that on my to-do list have a +KitchenRemodel as part of the to-do item. For example:

Todo Project

You can also include a “context” for the to-do item. This concept is derived (I believe) from Dave Allen’s GTD methodology where to-do lists are broken down into their proper contexts (@home, @work, @phone, etc.). In the example above, you can see I use contexts like @home and @errands. In fact, I only have a handful of contexts that I use with my to-do items: @home, @work, @errands, @freelance, @blog, and @mit. I’ll explain @mit shortly.

Finally, Todo.txt lets you set a priority for a to-do item. These are completely arbitrary, but are lettered, like (A), (B), (C), etc. I only use A, B, and C, and I use them as follows:

A = Today. This is something that I want to get done today.

B = This week. Something that I want to get done this week.

C = Someday. Something that I want to get done eventually. Mostly it’s on the list so that I don’t forget about it.

When using Todo.txt on the command line, it will sort and color-code your list by priority. So if I look at all of the items I want to get done this week, this is what I see:

Todo this week

Notice that they are sorted and color-coded by priority. The number at the very beginning of each line is the “task number” and the date is the date on which the item was added to my list. Todo.txt adds both of those automatically.

Adding items to my to-do list

I always have a console window open on the computer that I’m working on, so adding an item to my to-do list is very easy. Suppose I wanted to add an item to write a post on todo.txt. This is what I’d type at my command prompt:

t add "Write post on using Todo.txt @blog"

That item would be given a task number and date and added to my to-do list:

Todo Add

That’s all there is to it. I could have given it a priority or project as well simply by typing something like this instead:

t add "(A) Write post on using Todo.txt for +GP @blog"

If I decided I wanted to add or change the priority later, I could do that easily as well with something like:

t pri 60 B

which tells Todo.txt to change the priority of task #60 to B.

Todo.txt has iPhone and Android apps as well. I prefer the command line but sometimes the iPhone app can be handy if I am away from the computer. Here’s what my to-do list looks like on the iPhone app:

Todo iPhone
Todo iPhone

I store my todo.txt files in Dropbox and so they are accessible from any device to which I sync Dropbox, including my iPhone. That means I’m always working from a current version of my list, no matter where I am.

Continue reading

How I Automatically Capture Driving Data From my @automatic Link in a Google Spreadsheet

I have been using the Automatic Link in my Kia Sorento since December. It is a good little device that plugs into your car’s data port and pulls out all kinds of interesting information about your driving habits. For a while, you needed the iPhone app to browse the data, and the data itself was not extractable in any easy way, but no longer.

A while back, the Automatic tracker became available on IFTTT, with a bunch of triggers that can be used in automation workflow. One of those triggers is when a new trip is completed. So I created a recipe in IFTTT that logs the data of each completed trip to a Google Spreadsheet. For now, it logs all of the data, even though I might not use all of it. The data is logged within 15 minutes of completing a “trip” (going from point a to point b and shutting of the engine). Here is a list of the data that gets collected in the spreadsheet:

  • Car
  • Start Time
  • End Time
  • Duration
  • Distance (miles)
  • Average MPG
  • Fuel volume consumed (gal)
  • Fuel cost (dollars)
  • Hard brake count1
  • Hard accel count2
  • Duration over 70 MPH (minutes)
  • Duration over 75 MPH (minutes)
  • Duration over 80 MPH (minutes)
  • Trip Map URL
  • Start Location Longitude
  • Start Location Latitude
  • Start Location Map URL
  • End Location Longitude
  • End Location Latitude
  • End Location Map URL

The spreadsheet looks something like this:

Automatic Link

The great thing about this is that, like the FitBit Flex or my Google Writing Tracker scripts, the data is collected automatically. This is, in my opinion, of critical importance for personal analytics, because any time you have to take for manual actions lessens the likelihood you’ll continue to collect the data. For this data, all I have to do is drive.

I only have a week of the data so far, but it has already confirmed what we already knew: we have an incredibly good commute to and from work. I live about 5 miles from the office (5.18 miles on the roads according to the Automatic Link). When we leave the house at 7:16 am (as we did yesterday), we arrive at my office at 7:28 for a total trip time of 13 minutes. (Kelly has to then catch the Yellow Line from my office to her office in the District.) Coming home. Our reverse commute in the evening takes 12 minutes, despite being right in the middle of rush hour.

There are a few things I am trying to tweak with the spreadsheet. One downside is that the data/time is entered as a text field instead of an actual date/time and that makes some charting difficult, but I’m working on some code that will convert this automatically. Then, once I have more data, producing some charts and plots similar to what I’ve done for writing and walking should be easy.

One thing I’ve learned from this that I’d never thought much about before is the cost of our commuting into the office. Looking at the fuel consumption of our commute and Automatic’s estimated fuel costs, our commute costs us $1.85/day. That amounts to $9.25/week, or assuming we work 48 weeks out of the year, $444 in fuel costs commuting to-and-from work each year.

That number is actually high because there are days when we both work from home, but I suppose the number wouldn’t be less than $400/year.

I’m looking forward to delving deeper into this data once I have more of it to make it more meaningful.

ETA: I’ve embedded my IFTTT recipe for this automation below, for easier access.

IFTTT Recipe: Export Automatic Trip Data to aGoogle Spreadsheet connects automatic to google-drive

Notes

  1. The tracker detects when you brake too hard as part of its system for analyzing fuel consumption performance.
  2. The tracker detects when you accelerate at a rate that burns fuel in a less-than-optimal way.

My Technology Ecosystem, April 2014 Edition

I get questions, every now and then, about the technology ecosystem I use. I figure it’s pretty obvious from my posts, but in order to be perfectly clear, and in order to have a post I can point people to, I’m documenting my technology stack as of today, April 1, 2014. I’ll do this in layers from hardware, up through software ecosystems.

The Hardware Layer

Desktop Computer: Commodore 64

I recently upgraded from the VIC 20 to the Commodore 64, and I must say that it is a huge improvement. Having the additional memory is great, because I can now type in longer programs from the pages of BYTE Magazine. And, it’s got a 320×200 display, which is almost double what my old VIC had.

But an even better advantage of the upgrade has been…

Laptop Computer: Commodore 64

It is small enough to carry with me when I go places. I don’t even need batteries because this thing has a plug that I can snap right into a wall outlet. I mean, it’s cutting-edge.

Tablet: Etch-a-Sketch

The Etch-a-Sketch has been a life-changer for me and the family. It’s portable, requires no battery, and can do just about anything. I’d post a picture of our family Etch-a-Sketch, but it turns out we left it in the car (which my wife has driven to work). We use it a lot in the car to keep the kids entertained. They can watch their favorite movies on the Etch-a-Sketch, which I painfully draw for them, scene-by-scene, and which they then criticize. It’s a wonderful family-bonding experience.

Printer: My Royal QuietComfort DeLuxe Portable Typewriter

Granted, it is a little difficult to find toner ribbon for this device today, but when you do, it produces copy that harks back to the golden age of journalism, when newspapers were king, and on April Fool’s Day, columnists would wryly opine on how television was the wave of the future–nod, nod, wink, wink.

Truth is, finding decent typing paper is almost as difficult as finding ribbon, but I make due. Turns out it’s even easy to print multiple copies with this printer…

Copier: Carbon Paper

…because, carbon paper!!. This stuff is pure genius. Slip a sheet of it between two pages of typing paper, and type away. You come away with two copies of your document, each copying duplicating with precision every typo you make along the way.

Now let me discuss the software layer of my technology ecosystem.

Continue reading

How Many Steps Does It Take to Wear Out a Pair of Sneakers?

Yesterday, I noticed that my sneakers looked particularly worn out. Moreover, I noticed a significant hole in the left one. I discovered this hole while passing through a puddle. The insides are fairly worn as well, as are the bottoms. I took note of this mostly because it felt like I’d just purchased these shoes.

I don’t buy new shoes often, and I usually allow them to wear down. But this seemed unusually fast. It turns out that, time-wise, it was pretty fast, but wear-wise, these shoes have been through a lot, thanks to my daily walking.

I checked my Evernote timeline and found the receipt for my shoes, purchased on September 24, 2013, just under 6 months ago. I then checked my FitBit data and discovered that, in that six month period, I have walked nearly 1.84 million steps into these shoes. That’s about 900 miles worth of walking, day in and day out. If you consider that, on days where I can’t get my usually walking in, I only manage about 4,000 steps in regular activity, then you can see that these shoes would normally last me well over a year. But I tend to put between 15,000 – 20,000 steps a day in and so the wear and tear happens much faster.

My receipt from September indicates that I paid $64 for this pair of shoes. From that number, and the number of steps I’ve taken since, you can calculate some interesting rates. For instance, on a per-step basis, I paid 1/3000th of a cent per step. Looking at it another way, you can say that it costs me 7-cents to walk one mile in these shoes.

I am, of course, on the market for new shows. I’m looking for sneakers that are great for walking and are very comfortable as well. And if I can make them last to 2 million steps, then all the better. Any suggestions? Drop them in the comments.

2 Useful Insights I’ve Gained from Personal Analytics Data: Sleep and Productivity

In writing about personal analytics and data collection, one question I get more frequently than most is: what do you get out of it? Today I thought I’d share 4 insights I’ve gained into my own behavior from scrutinizing the data that I collect.

For those who haven’t been following along, I am fascinated by what data about our everyday lives can tell us about our behaviors. The data is often referred to as “personal analytics” and the movement behind this kind of data collection and analysis is called the “quantified self” movement. I collect data in four major areas:

Areas of Tracking

I collect data in other areas, too, but the key point about these four areas is that the process is entirely automated. I just go about my day, and this data is collected without any intervention or action on my part. I’ve already written extensively about my walking and writing insights so today I’ll focus on what I’ve learned about my behavior when it comes to sleeping and overall productivity.

1. Restless nights and sleep efficiency

You know those nights where you feel like you are tossing and turning all night long, getting very little sleep? Turns out, I do sleep on those nights, at least according to my FitBit, but my “sleep efficiency” is down below 90%. Here is a one recent example:

Sleep Efficiency

I’ve been capturing this type of data for almost two years now and I’ve learned a few useful things about my sleep habits by looking closely at the data.

  1. When my sleep efficiency is >= 95%, it feels like a restful night’s sleep. This is true for me almost independent of the number of hours I actually sleep. If I only get 5 hours of sleep, but my sleep efficiency is, say, 97%, I still wake up feeling like I had a good night’s sleep.
  2. When my sleep efficiency is between 90-95%, it’s a pretty good night, but the number of hours is more of a factor. If I get, say 7 hours of sleep with a sleep efficiency of 92%, I feel pretty good in the morning. On the other hand, if I get 5-1/2 hours of sleep with a 92% efficiency, then I don’t feel nearly as well-rested. According to the data, the time threshold is around 6 hours.
  3. When my sleep efficiency is less than 90%, I feel like I had a restless night’s sleep, regardless of hours actually slept.

I’ve been able to take this data and put together a chart of my sleep quality, based on two variables, sleep efficiency, and hours of actual sleep (vs. hours in bed).

Sleep Quality

I should not that I do not track how I feel each morning when I wake up. But on mornings when I felt particularly good or poor, I’ve checked it against the data from my FitBit and it is fairly consistent. For me, therefore, the above chart is a good representation of the quality of my sleep based on the two inputs.

How does this help?

Continue reading