Where do web standards come from?

Screen-Shot-2015-05-28-at-7.38.10-AM

I’ve spent the better part of the last six years thinking about where web standards come from.  Before joining USV, I was at the (now retired) urban tech incubator OpenPlans, where, among other things, we worked to further “open” technology solutions, including open data formats and web protocols.

The two biggest standards we worked on were GTFS, the now ubiquitous format for transit data, including routes, schedules and real-time data for buses and trains; and Open311, an open protocol for reporting problems to cities (broken streetlights, potholes, etc) and asking questions (how do I dispose of paint cans?).  Each has its own origin story, which I’ll get into a little bit below.

Last week, I wrote about “venture capital vs. community capital” (i.e., the “cycle of domination and disruption“) — and really the point of that talk was the relationship between proprietary platforms and open protocols.  My point in that post was that this tension is nothing new; in fact it is a regular part of the continuous cycle of the bundling and unbundling of technologies, dating back, well, forever.

Given the emergence of bitcoin and the blockchain as an application platform, it feels like we are in the midst of another wave of energy and effort around the development and deployment of web standards.  So we are seeing a ton of new open standards and protocols being imagined, proposed, and developed.

The key question to be asking at this moment is not “what is the perfect open standard”, but rather, “how do these things come to be, anyway?”

Joi Ito talks about the Internet as “a belief system” as much as a technology, and part of how I interpret that is the fact that it rests on the idea of everyone just agreeing to do things kind of the same way.  So, we don’t all need to run the same computers, use the same ISP, or be members of a common club (social network) — rather, all we need to do is adhere to some common protocols (HTTP, SMTP, etc).  No one owns the protocols (by and large) — they are more like “customs” than anything else.  It works because we all agree to do more or less the same thing.

So when we’re looking at all these new protocols appearing (from openname, to ethereum, to whatever), the question is not just “is this a good idea” but rather “how might everyone agree to do this?”. It’s a political and social problem as much as a technical problem.  And more often than not, there is some sort of “magic” involved that is the difference between “cool idea” or “nice whitepaper” and “everyone does it this way”.

Here is a crack at bucketing a few of the major strategies I’ve observed for bringing standards to market.  (These are not necessarily mutually exclusive, and are certainly not complete — would love to find other patterns and examples.)

Update: The Old Fashioned Way

Max Bulger makes a good point on Twitter that I have neglected here to include the traditional, formal methods of developing web standards — though standards bodies like the w3c and the IETF.  That’s how many standards get made, but not all.  For this post, I want to focus on hacks to that traditional process.

The Brute Force Approach

One way to bring a standard to market is to simply force it in, using your market position as leverage. Apple has been doing this for decades, most recently with USB-C, two decades ago with the original USB.

Screen-Shot-2015-05-28-at-7.33.18-AM

Word on the street is that USB-C was less of a consensus-driven standards body project and more of an apple hand off.  Time will tell, but now that USB-C is the port to beat all ports in the Macbook 12, it could become the single standard for laptop and mobile/tablet ports.  You can do this if you’re huge (see also: Microsoft and .doc, Adobe and .pdf)

The Happy Magnet Approach

I mentioned the GTFS standard, which is now the primary way transit agencies publish route, schedule and real-time data.  GTFS came to be because of work between Google and Portland’s Tri-Met back in 2005, as their collaboration to get Portland’s transit data into Google maps – so they created a lightweight standard as part of that.  Then, Google used “hey, don’t you want your data in Google maps?” as the happy magnet, to draw other agencies (often VERY reluctantly) into publishing their data in GTFS as well.   Here’s a diagram I made back in 2010 to tell this story:

Screen Shot 2015-05-28 at 6.26.48 AM

This approach includes elements of the Brute Force approach — you need to have outsized leverage / distribution to pull this off.

It’s also worth noting that GTFS won the day (handily) vs. a number of similar formats that were being developed by the formal consortia of transit operators.  I remember talking to folks at the time who had been working on these other standards, who were pissed that Google just swept in and helped bring GTFS to market.  But that’s exactly the point I want to make here: a path to market is often more is more important than a perfect design.

The Awesome Partner Approach

Not really knowing the whole story behind Creative Commons, it seems to me that one of the huge moments for that project was their partnership with Flickr to bring CC licensed photos to market — giving photographers the ability to tag with CC licenses, and giving users the ability to search by CC.

CC was a small org, but they were able to partner with a large player to get reach and distribution.

The Make-them-an-offer-they-can’t-refuse Approach

Blockchain hacker Matan Field recently described the two big innovations of bitcoin as 1) the ledger and 2) the incentive mechanism.  The incentive mechanism is the key — bitcoin and similar cryptoequity projects have a built-in incentive to participate.  Give (compute cycles) and get (coins/tokens).

While the Bitcoin whitepaper could have been “just another whitepaper” (future blog post needed on that — aka the open standards graveyard), it had a powerful built-in incentive model that drew people in.

The Bottom-up Approach

At our team meeting on Monday, we got to discussing how oAuth came to be.  (for those not familiar, oAuth is the standard protocol for allowing one app to perform actions for you in a different app — e.g., allow this app to post to twitter for me, etc).

According to the history on Wikipedia, oAuth started with the desire to delegate API access between Twitter and Magnolia, using OpenID, and from there a group of open web hackers took the project on.   First as an informal collaboration, then as a more organized discussion group, and finally as a formal proposal and working group at IETF.

From being around the folks working on this at the time, it felt like a very organic, bottom-up situation.  Less of a theoretical top-down need and more of a simple practical solution to a point-to-point problem that grew into something bigger.

The Pretty Pretty Please Approach (aka the Herding Cats Approach)

This one is hard.  Come up with a standard, and work really hard to get everyone to agree that it’s a good idea and adopt it.  It’s not impossible to do this, but it’s not easy.

This is more or less the approach we took back in 2009-12 with Open311.  In 2009, John Geraci from DIYcity (a civic hacking community at the time) wrote a letter to Mayor Bloomberg suggesting NYC take an open approach to its 311 system (I worked on the letter with John, as did several of my colleagues at the time).  From there, Philip Ashlock from OpenPlans took the lead on turning it into a real thing — working doggedly for 2 years with cities across the US, technology vendors large and small, and adjacent orgs like Code For America, to develop the specification and get it deployed.  As of 2012, there were something like 50 cities and 10 vendors live on the standard.   I would say that Open311 never had the “slingshot” or “magnet” it really needed to become huge and impactful — it was more of a slow grind.  But Phil in particular gets tons and tons of credit for making it happen.

And…?

In thinking about this, I also looked into the history of foundational web standards like HTTP and SMTP.

Here is Tim Berners-Lee’s original concept for an online hypertext system, and here is his more formal proposal to his bosses at CERN to fund initial work on the project.  He asked for $50k in manpower and $30k in software licenses.  Glad his bosses gave him the green light!

Here is John Postel’s original proposal for SMTP (the primary protocol behind email) to the IETF networking group.

I honestly don’t know the politics of how either of these went from whitepaper to real, and I’d love to hear that story from anyone who knows.

Another good story is HTML5, which was begun by a splinter faction away from W3C (dodging the slow process there and the focus on XHTML), and then eventually merged back into the formal W3C process.

One lesson

One big takeaway I’ve had from working on all of this is that these things take time, and that if you’re playing the open standards game, you need the ability to be patient (in addition to having a clever go-to-market hack).  It’s difficult to push a standard on a startup timeline. You’ll notice that many the historical players here had full-time employers (CERN, Google, universities, etc) that gave them the stability they needed and the flexibility to devote time to this sort of project.

And to reiterate the main point here — when looking at emerging standards and protocols, we’ve got to focus on the question “how do we get there”, and think hard about which go-to-market strategy to take.

// P.S., for a funny and slightly NSFW twist on this, see this post about the book “Where Did I Come From?” which I thought of when writing this post — my parents definitely read me that book when I was a kid and it made an impression.

Wanted: email apology bot

Maybe we all live in the email anti-Lake Wobegon, where we’re all “worse than average” at email, in our own minds. One problem with email is the giant guilt pile it creates — the psychological consisting of the number of emails you know are in there that you have forgotten about, ignored, or missed. My guess is… Read more »

Venture capital vs. community capital

Photo: Rudy (Loïs) Pignot I am in Paris this week for OuiShareFest, and spoke yesterday morning during the opening session.  OuiShareFest is in its third year as a large international gathering of folks interested in the peer/collaborative/sharing/networked society, put on by the community organization OuiShare. The topic of this year’s fest is “lost in transition”, and… Read more »

Anti-workflow: to-dos

A while back, I wrote about Anti-Workflow Apps — apps that solve problems for you without forcing you to adopt a workflow that you may never fully be able to adopt.  Workflow apps (CRMs, to-do lists, project management tools) are super hard to drive adoption towards, as everyone works differently and really resists this kind… Read more »

Dick Pics and Cable Company Fuckery

John Oliver has become the most important voice in tech policy (and maybe policy in general). His gift, his talent, his skill: turning wonky policy language that makes people glaze over into messages that people connect to and care about it. Last fall, he did took what may be the most boring, confusing term ever,… Read more »

Failure is the tuition you pay for success

I couldn’t sleep last night, and was up around 4am lurking on Twitter.  I came across an old friend, Elizabeth Green, who is an accomplished and awesome education writer — you’ve probably read some of her recent NYT mag cover stories, and it turns out she has a new book out, Building a Better Teacher…. Read more »

Financial Planning for the 90%

A few weeks ago as I was walking down Beacon Street in Brookline, I happened upon something amazing: The Society of Grownups. The Society of Grownups is a self-proclaimed “grad school for adulthood”, the idea is to give people the tools they need to manage their grown up lives.  The primary focus is on financial… Read more »

The Light Inside, The Fire Inside

Last week, a friend passed away after a relatively brief but intense battle with lung cancer.  I didn’t know Paul well, but he was very close with a few of my very close friends, and I had spent enough time with him to understand that he was special: he had a light inside of him.   A… Read more »

Regulation and the peer economy: a 2.0 framework

As part of my series on Regulation 2.0, which I’m putting together for the Project on Municipal Innovation at the Harvard Kennedy School, today I am going to employ a bit of a cop-out tactic and rather than publish my next section (which I haven’t finished yet, largely because my whole family has the flu… Read more »

Web platforms as regulatory systems

This is part 3 in a series of posts I’m developing into a white paper on “Regulation 2.0″ for the Program on Municipal Innovation Harvard Kennedy School of Government.  For many tech industry readers of this blog, these ideas may seem obvious, but they are not intended for you!  They are meant to help bring… Read more »

Technological revolutions and the search for trust

For the past several years, I have been an advisor to the Data-Smart City Solutions initiative at the Harvard Kennedy School of Government.  This is a group tasked with helping cities consider how to govern in new ways using the volumes of new data that are now available.  An adjacent group at HKS is the… Read more »

The magic of making hard things easy

I wrote earlier this week about how life is, generally, hard.  There’s no question about that. One of my favorite things about the Internet, and probably the most exciting thing about working in venture capital, is being around people who are working to re-architect the world to make hard things easier.  And by easier, I… Read more »

Everyone is broken and life is hard

That’s a pretty depressing and fatalistic post title, but I actually mean it in a positive and encouraging way.  Let me explain. It’s easy to go about your life, every day, feeling like everyone else has their shit together and that the things you struggle with are unique to you. But then, when you get… Read more »

Anti-workflow apps

“Workflow” apps hold so much promise.  Whether it’s a CRM, project management tool, to-do list, or some other tool, the promise in each case is to clean up our messy lives and help us be more organized and effective. The problem, though, is that getting people to adopt a workflow is really really hard.  That’s… Read more »

Finding Flow: writing vs. coding

When I first started to learn programming, about 15 years ago, I remember being surprised at how easy it was for me to get focused and stay focused.  I loved (and still love) the feeling of getting lost in a project, and could easily spend hours upon hours “in the zone”. No procrastination, no resistance, only focus… Read more »

Crowdsourcing patent examinations

Yesterday I spent part of the afternoon at a US Patent & Trademark Office roundtable discussion on using crowdsourcing to improve the patent examination process.  Thanks to Chris Wong for looping me in and helping to organize the event.  If you’re interested, you can watch the whole video here. I was there not as an… Read more »

Support services for the Indie Economy

Over the course of the past year, I’ve been interviewed a bunch of times about the “peer economy” or the “sharing economy” (Fastco, Wired, NY Times, PBS Newshour), with most of the focus on the public policy considerations of all this, specifically public safety regulations and the impact on labor. A question that comes up every… Read more »

The Professional Amateur

One way I have described myself is as a “professional amateur”.  I am both deeply proud and deeply ashamed of that.  Let me explain. For basically my whole career, I’ve been learning new fields and professions from the outside-in.  While I have an undergrad degree in Urban Studies, which ostensibly prepared me for interdisciplinary work regarding… Read more »

Half, not half-assed

My favorite book on product development and startups is Getting Real, published in 2006 by the folks at 37signals (now Basecamp).  If you haven’t read it (it’s freely available online), it’s essentially a precursor to The Lean Startup (2011). Back when I was leading a team and running product and OpenPlans, it was like my bible…. Read more »