My 2017 Twitter Year in Review: Small Data Edition

Note: This was migrated from my old Medium blog. I have decided to migrate from Medium to WordPress, and I’ll write about the reasons why soon, stay tuned.

Big Data has been a Thing™ for over a decade now. Google published the GFS paper in 2003 and the MapReduce paper in 2004. Hadoop started a few years later. Interestingly, the Google Trends trend for “Big Data” didn’t start to take off until 2011, though:

Why is that? Well, one clue is that Big Data grows along with, and is even eclipsed by, the rise of AWS, as this trend comparison shows (AWS in red):

That makes sense: Big Data took off once cloud infrastructure became cheap and flexible enough for the masses.

That said, not all data is Big Data. Some data is, well, Small Data. The Web abounds with thought leadership jockeying over the “big vs small data” question. For my purposes, the difference will be considered self-evident.

For me, Twitter is a perfect source of small data to analyze. It lets me reflect on my behavior in a structured, quantitative way. Such an analysis is complementary to the off-the-cuff reasoning that happens inside my head.

In 2017, I tweeted a whopping 21 times. In hindsight, I’d say my main reason for not tweeting much is what Facebook calls “meaningful interaction”, or lack thereof, in my case. Facebook just tweaked its News Feed algorithm again precisely to address this. I think they’re spot on.

With that hypothesis on the table, one straightforward solution suggests itself, before even looking at any data: spend more time engaging with my friends, and try to engage in tweet discussions around my interests, such as coding and design.

But let’s look at the data to be sure. First, I manually categorized my 2017 tweets by topic:

Date       Topic
01/07/2017 Design
01/08/2017 News
01/09/2017 Hamburger Eyes
01/13/2017 Design
01/20/2017 Startups
02/07/2017 Misc
02/14/2017 Hamburger Eyes
02/15/2017 Video Games
03/03/2017 Startups
04/10/2017 Apple Music
04/11/2017 Apple Music
04/13/2017 Misc
04/19/2017 Apple Music
04/27/2017 Apple Music
05/11/2017 Apple Music
05/17/2017 Apple Music
05/30/2017 Startups
06/22/2017 Music
06/28/2017 Music
07/25/2017 Apple Music
08/22/2017 Tech

Then, I used my blunderingly naive pandas and matplotlib skills to munge and plot the data:

import matplotlib.pyplot as plt
import pandas as pd

# Omitting a bunch of data and plot munging for brevity. Left as an
# exercise for the reader, or ping me on Twitter for advice.

data = pd.read_csv("2017_tweets.csv")
month_groups = tweet_dates.groupby("Month")
counts_by_month = month_groups.agg("count")

topic_groups = data.groupby("Topic")
counts_by_topic = topic_groups.agg("count")
counts_by_topic.plot(kind="pie", y="Count")

Here is the histogram by month:

Those results are somewhat funny. The January high of 5 is due to my renewed efforts at the start of every year to try and engage more on social media. Clearly, that effort loses steam as the year progresses. The matching high in April I’d guess was due to being done with taxes. Thus relieved and relaxed, I started listening to more music again. When I’m stressed, I tend to pick an album and listen to it on repeat.

Why no Twitter in the last 4 months of the year? Not sure. There’s a lot of circumstantial factors. I changed teams at work, for example. But nothing adds up convincingly. Let’s leave it as a small mystery for now. It’s more fun that way, anyways.

Here is the pie chart for the topic breakdown:

(Due to the height of my laptop screen, the pie ended up being an oblate spheroid #lol).

The biggest topic is Apple Music, when I share the song I am listening to straight from the Music app. None of these tweets drive interactions, but I like them as a form of self-expression, so doing that still makes me satisfied.

Another notable topic is Hamburger Eyes, a fantastic photography magazine I discovered over a decade ago. I should retweet their work more often. Also there is Design. I like having armchair design opinions, even though I do not practice.

What’s missing? Well, I had no tweets about coding. I like coding, more or less, but I haven’t really found discussing it on Twitter worthwhile. I prefer an in-depth book or article. As an experiment, I am now following more coding folks to see if this year will be any different. Also, no interactions with friends. Noted; will make a better effort this year.

Finally, my favorite tweet of last year is:

So there you have it.

On Courtesy

Note: This was migrated from my old Medium blog. I have decided to migrate from Medium to WordPress, and I’ll write about the reasons why soon, stay tuned.

I fly a lot — nearly every weekend — for personal/family reasons. It’s expensive, tiring, and disruptive in many ways.

There are small consolations, though. For example, when you fly enough, you get status with the airlines. On United, that means sitting in Economy Plus at no extra charge. The extra room is nice — I can work on a laptop without feeling like I’m sitting inside the trash compactor from Star Wars:

It’s also nice when your fellow passengers are courteous and well-mannered. I generally keep to myself, so I don’t fault anyone else when they do the same. I also understand that people can be brusque when they are tired or preoccupied. On the other hand, small gestures of civility are appreciated too.

My fellow passenger on tonight’s flight exemplified this good karma approach. When I was boarding and arrived at my aisle, I had to do the “excuse me, but I’m sitting in the middle” interruption. Most people will stand up, move, and leave it at that. However, this fellow stood up, made clear eye contact, and gave a friendly smile and a slight nod. It was like saying, “Sure, welcome to your middle seat! Here, let me get out of your way for a sec. Don’t worry, it’s no bother at all.” but without any words.

Similarly, at the end of the flight, when the plane had arrived at the gate, the seatbelt signed had turned off and everybody had started collecting their things to leave, this fellow immediately stood up and asked if I had any bag in the overhead compartment that I needed. I said that I didn’t, but he asked the lady in the window seat as well. They went back and forth a few times trying to confirm which bag was hers, but ultimately he located it and brought it down for her.

Not only is this a nice gesture to your neighboring passengers, it’s also a strictly more efficient way to deplane. Less time is wasted as each person gets up out of his or her seat, enters the aisle, blocks the entire flow of foot traffic, dislodges a bag, then resumes exiting the aircraft. Clearly, this fellow knows a thing or two about both manners and improving processes.

It doesn’t really matter who this person was. It could have been Joe the accountant or Jane the retired marketer. Courtesy isn’t about who you are, but how you act and treat other people in the moment.

In this case, the person was Stewart Butterfield, of Slack and Flickr fame. The only reason I mention this is as a data point for all the other aspiring entrepreneurs out there. Some use Steve Jobs as justification to be jerks. Please don’t, it’s sophomoric and counterproductive. You will likely be more successful by spreading good karma to everyone you meet.

I originally wrote this in early 2016. 2017 will be remembered for many things, including the opposite lesson — being an assbag to your customers on camera is a great way to, well, look like an assbag instead of a leader:

Originally posted on 22 Jan 2016. Revised 8 Jan 2018.

How to write good blog posts: my personal guidelines

Image courtesy of NASA Commons

Hello and welcome to my post on how to write good blog posts. I first wrote this in July 2015, but I last updated it in Feb 2019. Writing well, blog posts and otherwise, is an important topic for me, so I’m going to update this post for years to come, as I keep getting better at writing and blogging.

What do I mean by “good”?

There are many dimensions and layers to a good blog post. I can’t concisely focus on them all. Therefore, I want to focus on just one overarching goal for my writing, which is:

A good blog post has useful content, strong prose styling, and conveys some “realness” about the person who wrote it.

My overarching goal in blogging

If I can meet this goal, I believe that the other aspects (clarity, structure, etc) of a good post will fall in place over time. It might take me a few rounds of edits to get there, but I know where I’m heading.

Does this apply to all blogs?

My goal, and the guidelines I use to reach that goal, are stated broadly. They don’t imply any particular kind of blog.

That said, I generally don’t read blogs recreationally. Most of my leisure reading focuses on good ol’ books, magazines, and newspapers, albeit usually in digital form. Overall, I believe that blogs complement, rather than replace, those forms.

Rather, my main use of blogs is utilitarian. I look for blog posts to help me answer questions or figure out how to do some particular thing. As a professional software developer, I often look for “how to do X with code”. These days, the answer is usually found on Stack Overflow. However, ten or more years ago, much more of this information was on blogs.

Nevertheless, a utilitarian post is still a piece of writing by a person. Sure, my first priority is to answer my question or learn about some topic. But I am also interested in learning more about the person who wrote it. When it comes to software developers, I have found very few that are strong writers and express themselves as such, but there are notable exceptions, such as and Steve Klabnik.

Enough jibjab, get to the guidelines already!

Without further ado, here are my 5 guidelines on how to write good blog posts.

1. Be Yourself

Is it easy to convey your “real voice” in writing? If it is for you, fantastic. For me? Nooo.

In my experience, writing well is hard, and takes deliberate effort and practice to get better at. How you speak is often quite different from how you sound in writing (unless you are Seth Godin). Of course, your spoken and written voices will invariably differ. But most people are better at conveying their personality in everyday speech. The goal is to capture enough of that personality in your writing so that the writing reflects who you are.

And exactly how do you convey your personality in writing? There’s no single answer. Your personality is expressed by the kinds of stories you tell, the examples you highlight, word and idiom choice, etc.

Much of the world, billions of people, are on the Internet. Many of them write on Facebook and Twitter, and probably millions have active blogs. Therefore, chances are good somebody else is already writing about the same topics as you. However, rather than see this as a competition, consider it a chance to see the same topic from different viewpoints. That makes the world richer in content, rather than fiercer in competition.

2. Be Personal

Being yourself and being personal go hand in hand, but I keep them separate for emphasis. There are subtle differences to consider.

When I say “be personal”, I am not suggesting that you should divulge private details and the like. Privacy is important, and you should be aware of and in control of your own digital privacy choices.

Some people do choose to write online about very private topics, such as depression, grief, serious medical issues, etc. I’m sure it’s cathartic for some people to share these stories and for others to read them. However, the Web is an open medium. When you post anything online that is not anonymous, you must weigh the benefits of contributing to an open Web, with the risks, such as being hounded by vicious Internet trolls. Furthermore, true digital anonymity is really hard to get right. So be careful.

So, when I say “be personal”, I mean: focus on being authentic, sincere, and write from your lived experience. When done well, such writing can be fascinating, regardless of topic, and will resonate with many different kinds of people.

How can you be personal without divulging too much? This depends on the personal stories or topics you choose to write about. Nevertheless, you can be authentic and sincere on almost any story or topic, by using the right tone, nuance, and level of transparency.

3. Be Thoughtful

When you are talking to someone in person, it’s easy to “think out loud” and share half-baked ideas. The other person might be a close friend and understand the context, and help you refine your thoughts, play devil’s advocate, suggest alternative viewpoints, etc.

However, on the Internet, you don’t know your audience. You don’t know who is going to read your writing. Hasty, unsubstantiated ideas can be turned against you and used to mischaracterize or even defame you. Thus, being a responsible writer on the Internet requires being thoughtful in what and how you write.

One way I improve the thoughtfulness in my writing is by focusing on logical rigor. Good conclusions follow from strong, substantiated premises. Weak premises lead to faulty conclusions.

On the other hand, don’t take logical rigor too far and be some impassive Spock-like figure. What I really mean is: think deeply and carefully about what you say, do a responsible amount of research, and write in such a way to capture both what you are saying and why you are saying it.

Furthermore, focus on constructive efforts. It’s easy to poke holes; it’s much harder to disagree respectfully and offer credible, realistic alternatives.

4. Examples, examples, examples

Being yourself, being personal, being thoughtful: these all benefit greatly from examples!

Show, don’t tell. Especially when writing about software development or other practices that don’t lend themselves as readily to compelling, Moth-worthy stories. Joel Spolsky said it well in one of his classic articles:

The author had violated the number one rule of good writing, the “Show, don’t tell” rule. There was not a single story in the book. It was chock full of sentences like “A good team leader provides inspiration by setting a positive example.” What the eff?

Joel Spolsky, Introduction to Best Software Writing I (2005)

What’s more, on today’s Internet, it’s so easy to find examples—there’s almost no excuse for not doing it! Way back in the day, you had to go the library to find books or “encyclopedias” or old newspaper and magazine articles (on microfilm no less), then make photocopies, etc. Now, you just search and follow links. Factual? It’s usually on Wikipedia. Recent? It’s probably in your social media feed. The Internet doesn’t have everything, and Wikipedia and other sources are not perfect, but the examples you can readily find are often good enough to get the ball rolling.

5. Edit as Needed

The beauty of the Web as a medium is that you can edit as needed, five minutes or five years later. The beauty of editing is that it lets your writing become more you, more personal, and more thoughtful over time. You don’t have to “get it right” the first time you publish. Of course, you don’t want to be horribly wrong the first time either, but if you get the core idea right, you can iron out the details later.

That said, perfect is the enemy of good, so don’t let the opportunity to edit ad infinitum over-trigger your perfectionist tendencies. I usually let months or even years pass before editing an old post. But when I start editing, it usually takes me days of effort, and dozens of revisions, until I am happy with the new version.

Do these guidelines really work?

These guidelines are my North Star. I refer to them when I write new posts and when I edit older ones.

However, these are guidelines, not a straitjacket. When I write a new post, I focus mostly on getting the core idea right, and I check to make sure I haven’t gotten anything else egregiously wrong. As this article’s headline puts it, it’s about having a Bias for Action while avoiding the Oops Factor.

Furthermore, when I edit my old writings, I consider these guidelines even more carefully. The rush to get something published is over, so I can focus on quality.

In Conclusion

These are my guidelines on how to write a good blog post. And now for a parting quote:

I am for richness of meaning rather than clarity of meaning; for the implicit function as well as the explicit function. I prefer “both-and” to “either-or,” black and white, and sometimes gray, to black or white. 

Robert Venturi, Complexity and Contradiction in Architecture (1966)

Last updated Feb 2019. You can find older versions of this post on Github. And tap/click here to leave a comment. Featured image courtesy of NASA Commons.