The Public Data Layer

  •   Comments

I have been thinking a lot lately about the increasing importance of the “public data layer” — meaning, data that we will need (“we” applied broadly, meaning the general public, NGOs, government, scientists, journalists) to make sense of what’s going on in and increasingly busy, but increasingly quantifiable world.

First, some of the drivers here. In general, there is more data being generated than ever before, so much of which has a bearing on “public” issues.  A few of the specific drivers include:

  • Increasing role of “platforms” in regulated spaces (transportation, health, finance, education, etc) — these are enormous generators of data with direct and indirect bearing on public issues.
  • Sensors & IoT (publicly and privately owned) — same as above.
  • Abundance of media — as we have seen with the recent US election, the rise of social & independent media is democratizing but also problematic.
  • Personal health data — the cost of gene sequencing is dropping like a rock, which will lead to an explosion of health data. This data will provide personal value but can also provide enormous societal value.

Why this will be important?  Because all of these data have the potential to increase collective intelligence and societal knowledge.  And more specifically, we have the potential to redesign the way we make policy and handle regulation given these inputs.  If we do this right, we can get smarter at policymaking, and design regulatory systems that have both greater effectiveness and lower costs of implementation and compliance.

So, what infrastructure will we need to handle and process all of this public data?  This seems to be forming into a few broad categories:

  • Data pooling & analysis platforms — tools and APIs that make sense of these data — generic/foundational tools like Composable Analytics and Stae, and more specific, vertically-oriented projects & tools, like OpenTraffic and Aerostate.
  • “Regulation 2.0” platforms — specifically designed to facilitate a data-driven policymaking and regulatory process — for example, MeWe, Airmap, SeamlessGov.
  • Foundational and application-layer blockchains — on the pure tech side, this is the most interesting area of development.  Blockchains give us both public data access and data integrity in a way that’s not been possible before.  Much of the focus is still on “foundational” blockchains like Bitcoin, Ethereum, Tezos and Zcash, but eventually this technology will reach the application layer and we’ll have more explicitly “public” applications.  I also expect that Blockchains and Regulation 2.0 platforms will get ever closer and ultimately merge.
  • Mechanisms for identifying and amplifying truth — this is a tough, but important one.  We have two problems, in parallel: First, how to we discern truth from untruth?  And second: how do we give truth the attention it needs to “win”?  The big platforms like Facebook are experimenting with this now, and we’ll likely see more tools and services that help with this.

That’s the vision — where it seems clear that we are heading, and where we need to head.  So, the more important question is, how will we actually get there?  A bunch of questions/thoughts on my mind are:

  • Broad vs narrow?  Strikes me that we will see the most traction in narrow applications first — the thin edge of the wedge, that solves a concrete problem.  Also, the “personal data layer” hasn’t arrived in one broad platform either.
  • Open standards + distribution magnets: dating back to my work around open transit data, a key learning was that open standards need distribution magnets.  The thing that got transit agencies to publish data in the open GTFS format was Google Maps.
  • Portal access vs. real access — the natural tendency of data owners is to offer access via siloes and portals (e.g., Uber Movement).  This is something, but’s not the real thing — the more important question is how to get actual data moving.
  • Government isn’t the only audience: public data is of course useful for policymaking and regulation, but it’s equally important for scientific research and journalism.  These areas could end up being the initial leaders.

That’s it for now.  More to come.  For some more context on my thinking here, see Regulating with Data and Alternative Compliance.

Experience ↔ Design ↔ Policy

People often ask me how I ended up working in venture capital, and more specifically in a role that deals with policy issues (“policy” broadly speaking, including public policy, legal, “trust & safety”, content & community policy, etc.).  Coming from a background as a hacker / entrepreneur with an urban planning degree, how I ended… Read more »

Unintended Consequences

I’ve been struck recently by the power and surprise of unintended consequences. For example, a recent Slate article digs into flip side of the life-saving potential of automated vehicles: our reliance on car crash deaths for organ donors: “An estimated 94 percent of motor-vehicle accidents involve some kind of a driver error. As the number of… Read more »

Going after it

I’m in SF this week with the USV team – once a year we all come out here together, do a bunch of meetings and social events w our portfolio. Yesterday struck me — and it’s amazing how much of a surprise this is to me, after doing this nearly 5 years — with just… Read more »

The new normal

  •   Comments

The week before last, my in-laws were hit by a truck while crossing the street after dinner. The time since has been a disorienting whirlwind of sadness, fear, hope and thankfulness.  My mother-in-law suffered a very serious brain injury, and while she has cleared the first hurdle of basic survival, the outlook won’t be clear for… Read more »

Getting out the vote

Yesterday, a fabulous new tool launched — HelloVote: HelloVote makes it easy easy easy to register to vote.  Sign up w your phone number and do the whole thing over text. This is great for a lots of reasons — from its immediate practicality, to its more general lesson that it’s possible to build new,… Read more »

Alternative Compliance

  •   Comments

Summary To better support small businesses operating in regulated sectors, we should develop “alternative compliance” mechanisms — parallel regulatory regimes that achieve the goals of existing regulations but take an alternative, data-oriented approach to achieving them.  Such an approach would be especially friendly to the smallest of small businesses, and would take advantage of available… Read more »

Regulating with Data

  •   Comments

At this year’s Personal Democracy Forum, the theme was “the tech we need“. One of the areas I’ve been focused on here is the need for “regulatory tech”.  In other words, tools & services to help broker the individual / government & corporation / regulator relationship. In a nutshell: we are entering the information age, and… Read more »

Cable boxes, ridesharing and the right to be represented by a bot

Here are two tech policy issues that don’t seem related but are: the FCC’s current push to open up the set-top-box, and the lawsuits challenging Uber’s and Lyft’s classification of drivers as independent contractors rather than employees. The way to see the connection is through the lens of control vs. competition.  More specifically, they are… Read more »

Crypto debate: separating Security from Control

For the past few weeks, I’ve been following the FBI / Apple phone unlocking case, and digging deep into the debate around encryption, security and privacy. This debate is as old as the sun, and the exact same arguments we’re going through now were fought through 20 years ago during the first crypto wars and… Read more »

The Freedom to Innovate and the Freedom to Investigate

Earlier this week, I was at SXSW for CTA‘s annual Innovation Policy Day. My session, on Labor and the Gig/Sharing Economy, was a lively discussion including Sarah Leberstein from the National Employment Law Project, Michael Hayes from CTA’s policy group (which reps companies from their membership including Uber and Handy), and Arun Sundararajan from NYU, who… Read more »

Internet meets world: rules go boom

  •   Comments

Since 2006, I’ve been writing here about cities, the internet, and the ongoing collision between the two. Along the way, I’ve also loved using Tumblr to clip quotes off the web, building on the idea of “the slow hunch” (the title of this blog) and the “open commonplace book” as a tool for tracking the… Read more »

Big innovation and small innovation

  •   Comments

Yesterday at one of our bi-monthly team deep dives at USV, we got into the conversation of essentially “Big Innovation” vs. “Small Innovation”.  Those who have followed USV for some time know that at the core of the investment thesis is a belief in “decentralized”, “bottom-up” innovation — the kind that really became possible with… Read more »

Beam should have a hardware API

  •   Comments

We’ve got a few Beam telepresence robots at USV, and use them all the time.  Fred has written about them here.  We had a team meeting today, and we had two beams going at once — Fred and I were the first to arrive, and we were chatting beam-to-beam — he in LAUtah, me in Boston,… Read more »

Learning to skate

For the past few winters, I’ve been teaching my kids to ice skate.  Above is my son Theo at hockey practice a few weeks ago. At a certain point along the way, I got the bug and realized that skating was awesome and hockey was a beautiful sport.  So for the past year or so,… Read more »

Zero-rating: putting Net Neutrality to the test

It’s been an intense 10 months since the FCC approved its latest Open Internet rules (aka Net Neutrality). On the wired side, we’ve seen the unbundling of content, as channels such as HBO (via HBO Now) and ESPN (via Sling TV) have split from cable to go “over-the-top” with direct-to consumer offerings.  These are a direct result of the… Read more »

Hello, 2016

  •   Comments

Breaking the ice — been off the blogs for quite a while now. Looking forward to this year, the way I tend to every year.  2015 was a tough one for me personally — went through a bunch of shit on the family front that both demonstrated how tough life can be and also how… Read more »

As Massachusetts ponders ride-sharing regs, where’s the data?

  •   Comments

Today, hearings begin at the Massachusetts state house over how to regulate the budding ride-sharing / on-demand transportation industry (Uber, Lyft, et al). Adam Vaccaro over at has a good summary of the various competing bills — a pro-Uber bill that welcomes new Transportation Network Companies (TNCs) with relatively light-touch regulation, and a pro-taxi… Read more »

Supporting workers in the gig economy

  •   Comments

If there’s one thing I’ve learned throughout my years as a human, it’s that life is hard and people need help in order to make things work. That help can come in many forms: family, friends, co-workers, teachers, unions, healthcare providers, agents, assistants, coaches, therapists, strangers on the internet, you name it.  Point is, we… Read more »