Spec-ulation

Rich Hickey Deconstructs Semantic Versioning

beppu

7 minute read

https://www.youtube.com/watch?v=oyLBGkS5ICk

tl;dw

When Rich Hickey speaks, it’s a good idea to listen. He is able to see things that many of the rest of us cannot, and he is able to analyze and explain his thoughts with a precision that is rare even among programmers.

Little did I know that there was a problem with semantic versioning. It seemed like a reasonable system, and I never gave it a second thought until I watched this presentation where Rich exposed what the semantics of this system actually were.

Briefly, the semantics of x.y.z

  • z changes - don’t care.
  • y changes - don’t care.
  • x changes - you’re screwed.

That’s all this system can tell you.

Furthermore, this system implicitly formalizes software breakage by saying if you change x, you’re allowed to make backward incompatible changes.

However, do we have to break our software?

Rich points us toward another way.

0:00 Spec-ulation

Rich Hickey

0:58 This is not a talk about spec

It’s a talk about what spec is about

  1. Giving something to someone that they can use
  2. Making a commitment
    1. i.e. not taking it away later

2:46 Change

is it a thing?

3:35 ‘Change’

Origin:

  1. from Latin cambire “to exchange, barter”

You could change a cow into wheat!

  1. c.f. Eurogames

One-sided change is… theft?

  1. Anti-social at least

Productivity and soul-crushing at worst?

5:13

We want our software (esp. libraries/services to be different/better tomorrow.

What will that mean for our consumers?

5:42 Dependencies

[image of versioned library dependency tree]

6:59 But…

Artifacts don’t actually use other artifacts

There is nothing in the code about artifacts

7:45 Dependencies Redux

[image of versioned library dependency tree broken down by namespace] [exposes opportunity for dead code elimination; don’t need library z]

9:46 But…

Namespaces/packages don’t actually use other namespaces/packages

At least require/import appear in the code

Where is the mapping to artifacts? [no mapping of artifact to namespaces]

10:48 Dependency Truth (code)

[image of namespaces further broken down by function dependencies] [shows hat lib X is not a true dependency]

13:45 Do deps Force Versioning?

“What you can do is let Semantic Versioning provide you with a sane way to release and upgrade packages without having to roll new versions of dependent packages” http://semver.org/spec/v2.0.0.html

Is that what happens?

Often, no, cascading version bumps

  1. to let root ‘know’ about improvements to leaves, even if path nodes’ code unchanged

  2. level violation

15:27 Names, Levels, Scopes, Contexts

fns depend on (call) fns by name

ns/packages requires/includes set up a context in which those calls can succeed

Artifact deps/poms set up a context in which those requires can succeed

fn name scopes include ns but not artifact

  1. artifact context is MAGIC

17:19 Basis

Why do we put things in our deps?

  1. We need access to libs while developing

  2. Maven chases transitive deps

  3. Incorporated in artifact to communicate needs to our consumers

  4. “we tested against x,y,z”

But, course-grained, implies too much

  1. doesn’t capture actual deps, just context

19:49 (Ex)changes in Software :important:declarations:

What is required?

  1. fn - args

  2. ns - var names

  3. artifact ns/package names/paths

What is provided?

  1. fn - ret (proc/service effect)

  2. ns - vars/fns

  3. artifact - namespaces/packages

22:11 Growing Your Software

Accretion

  1. provide more

Relaxation

  1. require less

Fixation

  1. bash bugs

24:14 Breaking Your Software

Require more

Provide less

Unrelated stuff under same name

25:55 Change is Not a Thing

It’s one of two things

  1. Growth

  2. Breakage

In the small (fns), spec can help us determine which, prevent breaking specs

As long as we don’t try to do something silly like versioning specs!

27:56 Recognizing Collections

You only ‘change’ a collection by adding/removing members

Adding = growth, removing = breakage

Namespaces - collections of vars/fns

Artifacts - collections of namespaces/packages

Don’t conflate levels!

  1. My family doesn’t change when I put on a hat

(Everything intersting happens at the leaves.)

29:52 “Semantic” Versioning “Semantics”

1.2.changed

  1. “you don’t care”

1.changed.0

  1. “you don’t care”

changed.0.0

  1. “you’re screwed”

31:18 Even worse…

“you might be screwed”

Considered covering of change at all levels

  1. Good luck determining where

Might just as well change the name

32:26 Might as well change the name

32:28 MIGHT AS WELL CHANGE THE NAME

32:52 But…

that’s not change, That’s a new thing! Right.

33:09 Which name?

Levels again

33:39 Requiring More args? Providing Less on return?

i.e. incompatible spec

new function

old-ns/foo-2 or new-ns/foo

N.B - the namespace is port of the name

  1. ns aliases can ease transition

36:14 Providing Fewer fns/vars?

New namespace/package

37:20 Providing Fewer namespaces/packages?

New artifactId?

but… that’s what MAJOR segment is for?

  1. no, “any backwards incompatible changes”

    i.e. too-broad “semantic”

  2. The problem is artifact->namespaces magic means possibility of collision

    1. No ‘scope’ implicitly renaming children

[Rich was not sure what the right strategy for this situaion is.]

40:33 Doesn’t Doing the Right Thing Name-wise Make You Reluctant to Remove Things?

Yes. Good.

41:02 Breaking Changes are Broken

Full Stop

Don’t do it

Don’t try to figure out the best way to do it

Avoid breakage by turning it into accretion

  1. old and new can co-exist

42:25 So Maven is Broken?

Not really - how we use it may be broken

Maven central doesn’t let you ‘change’ artifacts

  1. and never ‘breaks’, is not versioned!!

    1. no “I’m using maven central 1234567.0.0”

You’re always hapy to use latest maven central

  1. Why?

  2. it’s an accreting collection of immutable things

44:27 (insert rotten sandwich image here)

46:29 So SemVer is Broken?

Yes

  1. It is, in part, about how to ship breakage

  2. and the other ‘semantics’ are of little utility

What instead?

  1. Maybe chronological versioning?

    1. YYYYMMDD.HHMMSS

  2. Conveys more and supports some forms of relativism

48:47 What about Git?

Wonderful, immutable, truth-of-the-code system, widely adopted

Content-based addressing

Almost completely ignored by artifacts/versioning, even though code basis

Deserves a role

  1. but, SHAs vs order/causality/readability

50:37 It’s a Social Thing

We won’t be albe to tech ourselves out of this

We need to agree that treating each other well is important

51:18 Local dev vs Open dev

Incompatible churn acceptable in private

Slack is not standup

OS user base is open and unknown

54:18 Coding for Growth

Open specs and data formats are key

Specs are about what you can do, not about what you can’t

Prohibition turns growth into breakage, cascades

Always presume you might be handed more than what you need or know about

  1. ignore, or have policy for it

58:31 What about Iterative Development?

Alphas are OK

But maybe should be in artifactId

Or - incremental API ‘publishing’

Open source is not an excuse for indefinite public thrashing around

1:00:14 The Only Truth is Runtime

Deps/POMs are just suggestions

  1. can be full of ‘conflicts’

Someone needs to build a classpath

  1. that alone determines runtime context

Possibly a lib set that none of the components have ever run against

1:01:59 Testing is Runtime Dependent

Plus, you can’t test against an open set of consumers

Artifact release testing is inherently limited

We could be reifying artifact sets at a macro (e.g. app or multi-app) level

1:03:47 (Live Coding Demo)

1:04:00 What about Web Services?

Same problems, same mistakes

  1. ‘versioning’ non-answer

  2. conflating collections w/contents

Web service is collection of ops

ops require/provide

Accretion could prevent a lot of client/service version hell

1:07:04 We Need to Bring FP to the Library Ecosystem

Currently update-in-place, excused but not corrected by versioning

  1. dependency hell == mutability hell

Makes programming fragile

Libraries less useful

  1. Even undesirable, whatever their features

1:08:19 “This is Impossible/Impractical”

Nope

Many examples of decade+ compatibility

  1. Unix APIs

  2. Java

  3. HTML

  4. Clojure?

Compatibility a prerequisite of success?

1:09:47 What If We Never Broke Anything?

names become enduringly meaningful

orthogonal compatibility checking becomes possible

find-grained deps can be explored

use the latest with impunity

compose with impunity

1:10:54 Open Challenges

We can’t ‘see’ some changes

  1. collection adds are easy

  2. fn args/ret not visible in source

    1. specs can help

spec compatibility context-dependent

  1. provide vs require

repo->artifact->namespaces not first class

tooling that reinforces the status quo

1:12:40 Opportunities for Clojure

spec

Flexible (vs fragile/brittle) dep awareness

Explicit code->artifact/repo support

First-class, fine-grained ‘publish’ for APIs

  1. ditto deprecation

Testing based on fine-grained deps

  1. generative testing needs this

1:14:23 Exchange > Change

Grow your software

Give birth to variation

  1. Don’t break, accrete

Think of the -children- consumers

Move forward without burning bridges

Create a lib/service ecosystem of which we can all be proud

comments powered by Disqus