2 Useful Insights I’ve Gained from Personal Analytics Data: Sleep and Productivity

In writing about personal analytics and data collection, one question I get more frequently than most is: what do you get out of it? Today I thought I’d share 4 insights I’ve gained into my own behavior from scrutinizing the data that I collect.

For those who haven’t been following along, I am fascinated by what data about our everyday lives can tell us about our behaviors. The data is often referred to as “personal analytics” and the movement behind this kind of data collection and analysis is called the “quantified self” movement. I collect data in four major areas:

I collect data in other areas, too, but the key point about these four areas is that the process is entirely automated. I just go about my day, and this data is collected without any intervention or action on my part. I’ve already written extensively about my walking and writing insights so today I’ll focus on what I’ve learned about my behavior when it comes to sleeping and overall productivity.

1. Restless nights and sleep efficiency

You know those nights where you feel like you are tossing and turning all night long, getting very little sleep? Turns out, I do sleep on those nights, at least according to my FitBit, but my “sleep efficiency” is down below 90%. Here is a one recent example:

I’ve been capturing this type of data for almost two years now and I’ve learned a few useful things about my sleep habits by looking closely at the data.

1. When my sleep efficiency is >= 95%, it feels like a restful night’s sleep. This is true for me almost independent of the number of hours I actually sleep. If I only get 5 hours of sleep, but my sleep efficiency is, say, 97%, I still wake up feeling like I had a good night’s sleep.
2. When my sleep efficiency is between 90-95%, it’s a pretty good night, but the number of hours is more of a factor. If I get, say 7 hours of sleep with a sleep efficiency of 92%, I feel pretty good in the morning. On the other hand, if I get 5-1/2 hours of sleep with a 92% efficiency, then I don’t feel nearly as well-rested. According to the data, the time threshold is around 6 hours.
3. When my sleep efficiency is less than 90%, I feel like I had a restless night’s sleep, regardless of hours actually slept.

I’ve been able to take this data and put together a chart of my sleep quality, based on two variables, sleep efficiency, and hours of actual sleep (vs. hours in bed).

I should not that I do not track how I feel each morning when I wake up. But on mornings when I felt particularly good or poor, I’ve checked it against the data from my FitBit and it is fairly consistent. For me, therefore, the above chart is a good representation of the quality of my sleep based on the two inputs.

How does this help?

Well, I’ve learned that there are things I can do to improve my sleep. I rarely aim for the “excellent” categories. I just try to fall into the “good” categories each night. I can occasionally control my restlessness by watching what I eat or drink before bed. Also, doing something mindless, like watching TV instead of reading off a screen can help.

This chart comes in handy on those days when I know ahead of time that I’m not going to get a lot of sleep. Maybe I have to be up early for a flight. On those nights, I will avoid reading off screens, and avoid caffeine well before bed. I find that on these nights, listening to audiobooks helps because it doesn’t involve looking at a page or a screen.

Over time, my sleep efficiency has improved and is often above 95%, which means I am almost always getting a decent night’s sleep, even on those nights where I get less than 6 hours.

2. Full Rich Days

Back in January, I began to use RescueTime to track my productivity, as well as how much time I spent on various things. For those who are unfamiliar with it, RescueTime is software that works on Windows, Macs, web browsers and other platforms and tracks what applications you use, what documents you use, and how long you use them. I then breaks them into categories, and whether they are considered productive or unproductive activities. It’s enabled me to track a lot of things that I’ve never been able to track before, and like the FitBit, it just works in the background. I don’t have to do anything. It compiles a “productivity pulse” for you for each day. Here, for instance, is my productivity pulse so far, for today:

The “productivity pulse” is similar in many ways to FitBit’s sleep efficiency. It is a measure of the percentage of time during the day that you were productive. You can see that for the hour or so I’ve logged so far today, my pulse is 68, which is pretty good for a weekend. You can see the categories over which my activity has been distributed. You can also see the breakdowns for each hour of the day. The red indicates time that I was very “unproductive.” This includes things like browsing Facebook, for example. I can drill into the data for any of these, getting down to exactly what document I was working with at a given time. But that’s not necessary for this examination. Much like sleep efficiency, I’ve found that certain productivity pulses result in certain feelings about my day.

1. Extremely-busy, non-stop days. When my productivity pulse exceeds 85 for a given day, it feels like I worked non-stop the entire day, barely coming up for air. This happens from time-to-time, especially when things are very busy at the day job. Here is once recent example of what a really busy day looks like.

I often feel completed exhausted after a day like this. And when I string together several days at a productivity pulse of greater than 85, I start to feel grumpy and overworked. I find that these days are the most difficult days for me to write, but I write anyway, which pushes the number up further.  Because of this, I try to avoid too many consecutive days at 85 and above.

2. Normal (“full rich”) days. My normal day seems to fall between a productivity pulse of 65-85. On days where it is closer to 65, I tend to have more meetings (and am therefore away from the computer more). When my productivity pulse false in this range, I feel like I had a good, productive day, but without feeling overwhelmed. A week at this level looks like this:

3. Lazy days. Whenever my productivity pulse falls below 65, I feel like I had a lazy day. This often happens on weekends, and I make exceptions for this because I’m away from the computer a lot more on weekends than during the week, and I only use RescueTime to track my time at the computer.

You can imagine a similar chart for productivity as for sleep, but the truth is that I haven’t yet collected enough data to where I can put such an outcome chart together with any degree of certainty.  What I’ve outlined above is as close as I can come, with the data that I’ve collected so far.

Perhaps the most significant thing to come out this is that I’ve cut way back on my realtime social networking. I’ve all but given up on regular checking of Facebook, for instance. And rather than live Tweet during the business day, I’ve shifted a lot my Tweeting to Buffer, which allows me to schedule things throughout the day so that I don’t have to take out time during the day to post the tweets.

If I can keep the “red” unproductive activities to a minimum, it allows me to focus on the more productive things.

Another thing I’ve found is that communication tends to take up large chunks of the day, and while things like email and calendaring are considered “productive” I’ve looked to reduce the time I spend on these by automating as much as I can. Some of this automation comes in the form of canned email responses. Some of comes from improved use of TextExpander. Some email I can’t avoid responding to, but I try to imagine having only the equivalent space of a post card to respond, thereby keeping things brief and to the point.

---

These are two ways that I’ve used the insights gained from personal analytics data to try to improve my behavior and performance. There are countless ways that you can do this, if you have the data. I always try to start simple.

The other thing I do is avoid too much analysis of data that requires effort to collect. I track my reading, for instance, but this requires manual actions on my part, so what I track is limited. I do not track food because I haven’t found an application that makes it simple enough to be both accurate and easy. If I have to (a) remember to track it and (b) take several steps to track it, I am not likely to do it. Food currently falls into this category.

That’s the great thing about the FitBit, RescueTime, and the scripts I’ve written to track my writing. I don’t have to do anything. Going through my normal daily activity collects the data automatically. And it is from that data that I can produce the kind of insights I’ve described here.