Post-Paperless Automation, Case Study #2: My Daily Almanac

I am currently away on an Internet Vacation. I’ll be back online on March 31. I have written one new post for each day of my Vacation so that folks don’t miss me too much while I am gone. But keep in mind, these posts have been scheduled ahead of time. Feel free to comment, as always, but note that since I am not checking email, I will likely not be replying to comments until I am back from my Vacation on March 31. With that said, enjoy!

---

[For my first post-paperless automation case study, you can read about my writing process and the automation involved therein.]

My desire to go paperless began with the challenge to myself: is a paperless office possible? I’d heard the term bandied about for a decade, but had never really seen a paperless office. So I decided to give it a try to with the help of Evernote, made it a reality, at least for me. Then I decided, if I can have a paperless office, can I get rid of paper everywhere? That led to my becoming Evernote’s Ambassador for Paperless Living, and my weekly Going Paperless series of posts. The more I went paperless, the more I began to see the possibilities of automation broaden. After all, if everything I wanted to keep was digital, it could be manipulated. And if it could be manipulated, it could be automated.

I am also a fan of the quantified-self movement (known in some circles as “personal analytics”.) I find the kind of data to come out of quantified self interesting to look at and contemplate, but also useful in adjusting my own behavior in order to improve myself. I’ll give an example of that today that deals with what I call my “Daily Almanac” generally, and specifically, how it has helped me with my fiction writing.

Before I get into that, it is important to understand that I take Evernote’s “remember everything” slogan seriously. Evernote provides a cloud-based platform that allows me to attempt to do just that. It is an organized repository of everything I can’t (or don’t want to) keep in my head. One goal I’ve had is to try to be able to recreate any given day of my life using Evernote. Of course, this only works for the days since I began using it a few years back. Each day, I gather all kinds of notes. Notes from meetings I attend, clipping from the web, notes for books I’ve read. Many of these notes are collected through automated processes, so I don’t even have to think about it. It just happens.

Recently, I’ve been working on what I call me Daily Almanac. This is a single note that gets sent to Evernote at the end of each day and summarizes those things that I did during the day that I am interested in keeping track of. But it does beyond this. It also looks for patterns and trends, lists those, and notifies me of performance milestones.

In its initial incarnation, there are 3 main areas which I want to be able to capture in this Daily Almanac:

1. Writing
2. Physical activity
3. Local weather conditions

These three things tell me a lot about what I managed to do during a day and under what conditions I did them. So far, I have implemented just the first into my Daily Almanac. (My time for writing code is limited these day). Let me emphasize that the key to this particular automation is that, beyond the initial writing of the code that does the automation), I don’t have to do anything. The data is collected, culled, analyzed and sent to Evernote automatically.

How I do this

My basic architecture is as follows:

• Use Google Docs/Scripts platform for the automation and metrics
• Use Evernote as a repository for the almanac.

Within my Google Drive, I have a folder called “Analytics” and within that folder, I have a number of Google Spreadsheets:

Each of these spreadsheets is updated nightly by a Google App Script I have written. For instance, the “FitBit Daily Data” spreadsheet is updated by a Google App Script each night. This script uses the FitBit API to pull in my FitBit data for that day. (The FitBit Stats spreadsheet pulls in minute-by-minute data as opposed to a daily summary.) My “Writing Data” spreadsheet is updated nightly by a script that does 2 things: (1) gets the word count for fiction/nonfiction that I’ve written today and (2) get the word count of blogs posts I’ve written today.

There is also, you’ll note, a Scripts folder, and that folder contains Google App Scripts that I’ve written that run independently of the spreadsheets. For instance, I have once such script called my “Daily Almanac” script. At present, what this script does is collect information about what I’ve written in the day, and sends a resulting summary to me and to my Evernote account. Here is what the summary looked like on March 12:

You’ll note that the message summarizes my total writing for the day (2,523) words and then breaks the summary down into 2 sections, Fiction/Nonfiction-writing and Blogging. You’ll also note that the message reports any streaks I have going, and when I set a new record. This has become particularly useful to me, keeping me writing everyday because I don’t want to break the streak. Seeing this email each morning is a motivator for my writing that day. Not shown in this example, but something else that the script does is report any record I set for writing length on a given day. My current fiction-record since I’ve been using this Almanac is 1,650 words. If I were to write more than that on one day, the fact would show up in my Daily Almanac for that day. Here is my Almanac from March 13:

 

In this case, you can see I continued by fiction-writing steak, extending it to 15 days. But I didn’t do any blogging on March 13, and so my streak ended there.

This contains just writing information right now, but I am actively adding other sections to the daily almanac. One section will contain a summary of my physical activity for the day. The source of that data comes from my FitBit device. But the presentation in the almanac will be similar. It will list how many steps I walked, how many flights of stairs I climbed, and it will also provide record information (“Today was your best day ever!” or “You’ve walked more than 10,000 steps for 10 consecutive days now.)

I’m also adding a section for weather conditions. This data comes via the WeatherUnderground API.

All of the raw data resides in Google Spreadsheets and I use Google Scripts to automate the collection process, as well as the process of generating and sending out my almanac each day.

My two big benefits

1. I don’t have to do anything. Once the code has been written, I never have to think about it again. The automation happens behind the scenes, the emails comes to me and Evernote, and should the script fail for some reason, I am notified of the failure.

2. I have some great data in Evernote. Now, when I search for all of the notes created in a given day in Evernote, I have a much sharper picture of what has happened in the day thanks the Daily Almanac note. And because of the writing automation I’ve written about in an earlier post, not only can I see that I wrote 500 words on March 11, but I can see exactly what those 500 words were.

Are the scripts available?

I imagine that a few people will ask if these scripts are available. Not yet, but they will be. My time is extremely limited, and my priority in the day is to get my writing done first and foremost. If there’s time, I do some coding, but that certainly isn’t every day. What I have done so far is created a repository in GitHub. I’m in the process of cleaning my scripts so that they are more usable to people other than myself. Once that is done, I will share the repository on GitHub and announce that it is available.

---

Over time I plan on adding more to my Daily Almanac, going beyond writing, walking and weather. But for now, this is a good start and provides a solid architecture for moving forward with my quantified-self hobby, my desire to go paperless, and my wont to remember everything.