localfirst.fm

A podcast about local-first software development

Listen

Conference

#1 – PVH: An Intro to Local-First

In this inaugural episode, I'm speaking to Peter van Hardenberg, who helped to coin and popularize the term Local First. As the director of the Ink & Switch Research Lab, he's been on the forefront of this work for the better part of a decade. My conversation with him today starts with the basics of what Local First is and why you, an application developer, should care about it.

Mentioned in podcast:

Links:

Thank you to Expo and CrabNebula for supporting the podcast.

Transcript

Introduction
00:00The most important thing about the cloud is collaboration.
00:02You can send someone a link and you're working together right away.
00:05But the downside of the cloud is that you don't actually have the software.
00:09And so what we think you should have is a copy of the program and the data on
00:14your computer, but it should still be able to collaborate with other people.
00:18Welcome to the localfirst.fm podcast.
00:21I'm your host, Johanna Schickling, and I'm a web developer, a startup founder, and
00:25love the craft of software engineering.
00:27For the past few years, I've been on a journey to build a modern, high quality
00:31music app using web technologies.
00:34And in doing so, I've been falling down the rabbit hole of local first software.
00:39This podcast is your invitation to join me on that journey.
00:42In this inaugural episode, I'm speaking to Peter van Hardenberg, who helped to
00:47coin and popularize the term Local First.
00:50As the director of the Ink & Switch Research Lab, he's been on
00:53the forefront of this work for the better part of a decade.
00:57My conversation with him today starts with the basics of what Local
01:01First is and why you, an application developer, should care about it.
01:05Before getting into it, also a big thank you to Expo and Crab Nebula for
01:10supporting this podcast and supporting the local first ecosystem as a whole.
01:15And now my interview with Peter.
01:19Hello, welcome Peter.
01:21Thank you so much for making it today to kick off this new podcast with me.
01:26So would you mind introducing yourself?
01:28Yeah.
01:29Hi, my name is Peter Van Hardenberg.
01:31I am the director of Ink & Switch Research Labs.
01:34We're an independent industrial research lab studying the future
01:37of computing and tools for thought.
01:40Awesome.
01:40Peter and I have known us for quite a while.
01:43Back then, Peter, has been just closing up his last chapter at Heroku
01:48which I've been always a big fan of.
01:50Peter found his way now working on local first and defining what local first is.
01:58Before going into it, what has led you to local first?
02:02Yeah, I mean, obviously it's a bit of a departure, right?
02:04Like, here I was building a platform as a service.
02:07You know, Heroku, for those who don't know, is a, basically a host
02:12Primarily when it started for Ruby on Rails applications, but over
02:15time for pretty much everything.
02:17And the idea behind Heroku is that you could build an app on
02:20your machine and then push it.
02:22To a continuous delivery system, and it would put it live in the cloud, and there
02:26are other systems like that today, you can imagine it as you know, proto Netlify
02:30or early kind of, containerized cloud.
02:35Yeah, and it is a bit of a departure to go from trying to get everybody's
02:38apps running in the cloud to telling people that maybe they should
02:41just run them on their computer.
02:44So I guess I should really explain how we got there.
02:47I think for me.
02:50I'd like to tell a story about riding the train in San Francisco, the subway there.
02:57And my friends had been working on a music app called Rdio which was
03:00sort of a competitor to Spotify.
03:02They're not around anymore.
03:03And you know, as I was riding on this train, I had been listening
03:08to music in the online mode when the train was above ground.
03:11And then when we went into the tunnel, the music would just stop working.
03:15And it was really...
03:17upsetting to me because like I couldn't go back and even listen to songs I'd already
03:22listened to or like scroll through the playlists that I had because I wasn't in
03:27offline mode and I just remember having this feeling like we've really blown it
03:35we've turned software from this thing that you can just have like they call it an
03:39informational good that means that like you know the marginal cost of copying it
03:43and using it as zero it's just it was this idea that like software had gone from this
03:47thing that anyone could have a copy of To a thing that didn't even work on the train
03:51because I didn't really have the app.
03:53Even though I'd had the data in my hand on my screen in front of me, in
03:57my ears listening to it, somehow it had just completely evaporated in the time
04:01it took a train to go into a tunnel.
04:02right.
04:03And riding a train is not this out there thing.
04:06It's like this is a daily thing.
04:08Yeah, is my daily commute.
04:09And, you know, there are people out there who say, Oh, yeah, you know, the
04:12Internet's going to be everywhere next week, and have you taken a subway lately?
04:16You know, and like, both in big cities, right?
04:20You have this problem, but also in more rural communities, right?
04:22There's lots of places where there's no Internet, or the Internet is slow.
04:26And you can even be in places where there is Internet, right?
04:28But your cell phone reception isn't great next to the fridge,
04:32or your Wi Fi has a cut out.
04:33Faced this every day when I'm just like leaving the house and there's just this
04:37little gap of the wifi stops responding.
04:40There's a little bit of signal and my apps just don't work properly anymore.
04:44Yeah, everything kind of locks up.
The Problem with Current Software
04:50And so that got me thinking, like, well, why do we build
04:53things the way we build them?
04:54And the answer to some extent, I think, was just that, you
04:57know, we told people to.
04:59Right.
04:59We made it easy to build apps for the cloud, and then everybody did.
05:03Yeah.
05:03Yeah, and so at Heroku, I had worked on our data services mostly.
05:07That was our Postgres service.
05:09And I started thinking about how really the problem is that the data
05:13that you have for your application doesn't actually live on your device.
05:18It lives on the server.
05:20And so I started thinking, surely there's some way to get this data
05:23from the network down to the computer.
05:26And it's funny.
05:27You know, Johannes, you and I met originally through Prisma
05:31and GraphQL and database schemas and those kinds of things.
05:34And this always struck me as a really adjacent problem, which is
05:36that fundamentally, on your server, you're doing this process of taking
05:40data from a database and getting it into your sort of API back end.
05:44And then you do a separate, distinct process to get data from
05:47the API to the client, right?
05:50And then when it gets to the client, you just kind of throw it away.
05:54And so I started thinking like, well, maybe it would make a lot more
05:58sense if we just sort of synchronized data between those points instead of
06:02fetching it on demand all the time.
06:04And in fact, that's sort of how offline modes already worked.
06:06And so I started thinking more and more about this and got involved with
06:11some great fellow researchers like Martin Kleppmann, who's currently at
06:15TU Munich, but was at Cambridge at the time and had just written a paper about.
06:19What are called JSON CRDTs, which is like a technology for synchronizing
06:23changes between two documents in a way where you didn't just
06:25have to clobber one or the other.
06:28And yeah, that one thing led to another, and we've been working on
06:31the problem for about five years now.
06:33I think you've been telling me about it already many years ago.
06:37And initially I wasn't quite sure it's like, Hey, is this about only
06:40offline mode, but step by step I realize, Oh, there's much more to it.
06:45And like, fundamentally this can unlock much higher quality software
06:49and also not make it harder for me as an application developer, but
06:53actually make things a lot simpler.
06:55We're maybe not fully there yet.
06:58But that's a vision I totally buy into.
07:01So this is the path that has led you to think what becomes local first.
07:07Did you coin that term?
07:09So the term local first came around in a call we had one day.
07:14I think...
07:15Folks have given me credit at the lab, but it was definitely one of those
07:18open ended conversations where several of us were kind of trying to find the
07:22right term for this for a long time.
07:24I'd actually planned to refer to this as serverless technology,
07:28but then Amazon came out with this so called serverless technology,
07:33which in fact was just all servers.
07:36So I found that sort of like frustrating and slightly upsetting
07:39because it felt misleading.
07:41But we really like the idea of calling it local first technology because
07:44the idea was to emphasize that it's not about doing away with the cloud.
07:48You know, it's not really serverless.
07:50What we wanted to say was actually, we just want to prioritize the user's
07:54experience on their device and make that easy to deliver on high quality.
07:58Yeah, I like that a lot.
08:00You've been mentioning the story of Listening to Rdio on the subway and that's
08:05not working, but from an application developer perspective what would you
08:09say were the frustrations that has led you to think about local first.
Complexity of Modern Web Development
08:14Yeah, I mean, it's a few things.
08:15One is if you just think about software development as an app developer, it's
08:20sort of obscenely complicated, right?
08:23You've got To build a client app, probably using Node and JavaScript
08:29and React and Vite and tailwind and all these technologies.
08:35Then you have to build like the API backend, which is its own separate kind
08:39of node app or part of your existing app with all of its own sort of like JWTs
08:47and, you know, database sync technologies and GraphQL and, you know, you're
08:52building out all that whole system.
08:54And then behind that, you actually have your data layer, which is like.
08:56Postgres and so you have like in kind of the best case in the minimum case.
09:02You know, for of modern web dev, you've got this three tier stack as
09:05they call it, but the reality is, it's not just those three tiers.
09:08You also need like pager duty for letting you know when things are down.
09:12And maybe you're using like a bunch of weird Amazon services.
09:15You got a DynamoDB thing going on.
09:17Then, you know, No one's hosting things on Heroku anymore.
09:21You got to put stuff up on, you know, you need a Kubernetes cluster somewhere.
09:24And, you know, it's just, it's a mess.
09:27it's insanely complicated.
09:28So there's just like a lot of complexity feels gratuitous for like making an
09:34app for your soccer team or something.
09:35And not even from a tooling perspective, but just the minimal
09:38architectural footprint is quite a lot for often very simple apps.
09:43That's right.
09:44And then on top of that, you have to pay for it all.
09:46Right.
09:47You know, maybe you're a YC startup, you get some credits to get you
09:50started, but that's definitely the drug dealer model, right?
09:52Like first one's free, right?
09:54And then, you know, I have personally known a number of companies that died,
09:58even though they were, uh, their product was growing rapidly, but just the cost
10:05of running their infrastructure was too high to justify the system, right?
10:10Like they didn't see how they'd be able to turn this thing into a Like a venture
10:14backed billion dollar valuation company.
10:16And so they couldn't get any money to keep the lights on despite having
10:20a popular and growing product.
10:21So that's crazy.
10:23And then like.
10:24Even on top of all of that, like, let's say that you've managed to, you know,
10:28raise enough money to hire an entire team to be on call for you and you managed
10:32to, you know, build a product that you can convince somebody will be a billion
10:36dollar, you know, valuation business at some point and you learn all of those
10:40skills and you build all those systems out and everything, you know, ultimately,
10:45it's still very fragile, right?
10:47In In time, as well as in space, you know, we've all had all of these
10:51products now that we loved and used, whether it's, you know, Google reader
10:56or dark sky or, you know, pick your favorite favorite dead tech, you know,
11:00and these products go on the founders, when they get acquired, they put up
11:04this blog post about what an incredible journey building this product has been.
11:08And, you know, the incredible journey that's happening is that
11:10your software with your data is going away and you can't have it anymore.
11:15Right.
11:15They're being acquired.
11:16They're shutting everything down.
11:17And if you're lucky, you know, as with Rdio you can
11:20download a bunch of JSON files.
11:22That's all that's left for you as the user.
11:23And I just want to say, like, this is a problem we created for ourselves, right?
11:28Like, I can go and download an old Microsoft, you know, MS DOS copy of,
11:32like, WordPerfect or something from the late 80s, and it'll still run.
11:38You know, I can get doom on a floppy disk and it'll still run, but like
11:42software that came out for the cloud, you know, it's just gone when it's gone.
11:46And that's it, right?
11:47This is, they call a certain period of history, the dark ages, not, you
11:53know, formally because they were bad, but because they're dark to us today,
11:58because We can't go back and look at the records from them, they were lost.
12:02And so similarly, we have built a dark age for technology.
12:06Not that it's bad, there are a lot of things that are great, but that there
12:09won't be a record of our work here, right?
12:12Like the things that you and I are building, the work that
12:14I did at Heroku, the apps that we've made and put out there.
12:18Because they're not, they don't exist outside of the one set of servers they
12:23run on, you know, when they go offline, that, that's it they're gone forever.
12:28And I think that kind of sucks.
12:31Yeah, I fully agree.
12:32I mean, having gotten into software development, step by step, I
12:36could take my first steps with building a little thing with HTML,
12:40CSS and step by step, you want to make it more functional, more real.
12:43And then you start pulling on that thread, like all that complexity comes
12:48towards you and it's never ending.
12:50So it's so much to learn, so much to maintain, so much to pay for and operate.
What is Local First?
13:00So maybe this is a good point to ask, what is local first?
13:05What is Local First?
13:06local-first software is software that runs on your computer and
13:11collaborates with other people.
13:12We think that the most important thing about the cloud is collaboration.
13:16This ubiquitous, always accessible, copy of your data from anywhere.
13:20You can send someone a link and you're working together right away.
13:23Nothing to download, nothing to install.
13:25But the downside of the cloud is that you don't actually have the software.
13:29And so what we think you should have is a copy of the program and the data on your
13:34computer, but it should still be able to collaborate with other people, right?
13:38We want the good parts of the cloud combined with the good
13:41parts of the old way of building.
13:43And in a way right now we have great collaboration tools like
13:48Google Drive and Figma, etc.
13:51But it's all by default lives in the cloud.
13:54So you suggesting let's have the cake and eat it too, so that the software
13:59fully works on the client and we only use the server for the absolutely necessary
14:05pieces but still have collaboration.
14:09That's right.
14:09And Doug Engelbart, you know, a famous computer science
14:14researcher he talked about how we can de- augment our intellect.
14:20We can make things worse for ourselves.
14:23And he demonstrated this really poetically by taping a brick to a pencil
14:27and asking people to write with it.
14:29So I'm not here to say that there are free lunches necessarily in solving these
14:33technical problems, but I do think that we may have taped a few giant bricks
14:37to our pencils here, and that there's a lot of unnecessary difficulty that
14:41we've introduced by picking architectures that are not well suited to the task.
14:47These are in many cases, great technologies.
14:49It's just that we're deploying the same technology that we would use to
14:53build a billion person app to build a 10 person app or a million person app.
14:58And the requirements are just radically different.
15:01Now, I think this sort of homogenization of the technical stack is part of why
15:08we're in the position that we're in.
15:10Like Jonathan Edwards famously said, you know, we were
15:13promised bicycles for the mind.
15:15But they gave us aircraft carriers instead, right?
15:18So, so we're building aircraft carriers when we need bicycles.
15:23That doesn't mean that the aircraft carrier isn't useful.
15:25It's very useful if you need to project force in the South Pacific as the U.
15:31S.
15:31Navy, but it's not terribly helpful if you want to go get groceries.
15:35Right?
15:35It's the wrong tool for the job.
How to build Local First Apps
15:40So you've mentioned that it basically should work fully on our device
15:44without the server and the server augments that software by giving
15:48it additional capabilities, such as collaboration, maybe it also helps
15:52with automatic backup, et cetera.
15:55So, that.
15:56is very clear.
15:57However, it would not be super clear yet how I would go from typically
16:01building a three tier web app to building things in that model.
16:06My intuition would be maybe adding tons of caching, et cetera.
16:11Would I still have that server database?
16:14So what is the typical way how I built local first apps?
16:17Yeah, you kind of have two main problems.
16:19There's the program and the data, right?
16:21We'll just kind of boil it down.
16:23The program in some sense is the easy problem to solve.
16:26So, you know, if you're on the web, what you want is a PWA, progressive web app.
16:31You're going to use a service worker to cache the code.
16:34You're going to try and run everything in the browser, and you're going to
16:37keep it stored so it works offline.
16:40If you're on mobile, in many cases you already build this way,
16:44Yeah.
16:45Like, everything that Apple publishes first party, more or less.
16:49Is already local first, right?
16:51Like Apple notes.
16:52It's not like you ever pull out your phone in the elevator to, like, make a
16:56note from a conversation you just had.
16:58And all you see is a little spinner.
16:59No, that's ridiculous, right?
17:01Like Apple knows,
17:02I think Apple notes is a great example of what an app should feel like, whereas
17:06I do a lot of times see that spinner on the hallway where I don't have perfect
17:10internet connectivity in other apps.
17:13right?
17:13Because you're at somebody's office and you know, want to make some notes about a
17:15conversation not on the WiFi or something.
17:18And you're like, Oh, God, I gotta
17:19wiFi password?
17:21Yeah, no, that's ridiculous.
17:22I'm just trying to take a note here.
17:24Okay, so, so you need to get the program on the device.
17:26That's one.
17:27And then you need to get the data on the device.
17:29That's too.
17:30So how do you get the data on the device?
17:32Well, in some very limited cases, you just put the data into the program, right?
17:36So like, I don't know, maybe it's a reference for a board
17:40game or something like that.
17:41You can just download the data and leave it in the app.
17:45But of course, that's not like the interesting case.
17:47A lot of apps, most apps, I think, you know, you have user generated content.
17:51That's why you're there, right?
17:52It's a map, it's a note, it's whatever you're there to do.
17:55And so in that case, what you need.
17:58Is some way to get data from the cloud to the device and then back to the cloud.
18:02And the tricky part is you need to deal with the fact that people can go offline.
18:07And so that data is basically some data I might've previously created.
18:12For example, if it's the Apple notes app, then maybe some notes
18:16I've created on my Macbook.
18:18I think we don't need to talk too much about like how do you deliver the program.
18:23But the data aspect that seems really tricky.
18:26So what are the problems that you've seen?
18:29And what are the current best practices or different way how we can architect things?
The Data Is The Hard Part
18:34Yeah.
18:34And again, I like anchoring this in , how did we get here?
18:37We've been downloading data for a long time.
18:38You can get a CSV file off the Internet.
18:40You can email somebody in Excel file, right?
18:43We kind of get that, like, then you have it.
18:45You know, I can email you an Excel file and then you can
18:48open it like no big deal, right?
18:50That seems easy.
18:51Similarly, like with my with my story about the music app, you know,
18:57I could download those playlists and just cash them on the device.
19:01And so that's like a kind of a modest thing you were
19:03saying, just cache everything.
19:04Cool.
19:05Okay.
19:05As long as all you're doing is reading, no big deal.
19:09That's fine.
19:10Right?
19:10That just works.
19:11Cool.
19:11We'll cache everything.
19:12The problem is that we also still want to be able to work when we're offline.
19:17Right?
19:18one our tenants is you should never, ever be stopped from doing what you want to do.
19:24By the availability or lack of availability of another computer.
19:28And I would say this is maybe also where you can draw a line between.
19:32Current, not yet local-first software that's not entirely useless in an
19:36offline scenario if it caches things a lot, so at least I can read it.
19:41But to go from there to that, it allows you to still do
19:45the job completely offline.
19:47I think this is sort of like a barrier that is incredibly hard to overcome with
19:52like your more like typical three tier app
19:55optimistic,
19:56of caching.
19:57yeah, exactly.
19:58And it's not only that, but, like, it's just complicated to think about, right?
20:02Like, there's a lot of subtle problems.
20:04Oh, if you know, you click the OK and then it goes green right
20:07away, but really you're running the fetch in the back end and
20:11have an
20:11post request failed.
20:13now we've already told the user that it works.
20:16Okay, now we need to like raise an exception.
20:19Let's make sure that we check if we're online first.
20:21Oh, but the browser thought we were online.
20:23But now it's just, it's a nightmare as let's not do
20:26I've seen many times that basically like shows you indicates, Oh, this was great.
20:31And then you see like a little spinner, a pop up, something has gone wrong.
20:36Please reload the
20:37Refresh the browser.
20:38You've lost all your work.
20:39One of my all time least favorite messages is on Google Docs.
20:44Sometimes if you work on a document, like while you're on a flight,
20:47even though you've downloaded it, remember to add the extension, cached
20:51it before you got on the plane so that it would be available offline.
20:55And then you edit the document on the flight and then you land and
20:57you come online and it says, we're sorry, this document has changed
21:01too much while you were offline.
21:04We're now resetting you to the cloud state and then all your data is gone.
21:10That's only happened to me once or twice, but man, that hurt.
21:14What we do is we can, you know, if it's just you working again, there's
21:19not really any problem here, right?
21:20Like I can edit the data.
21:22And then just when I come online, we upload it.
21:24The real challenge is if you have two different things, editing
21:27the data, and then you need to merge their changes, right?
21:30So like I added a track to the playlist.
21:34And, you know, Spotify added a track to the playlist.
21:38Okay now we have a conflict.
21:40Conflict is the technical term we're going to use here.
21:42And so that's the hard thing.
21:43And lots of different systems have approached this in different
21:46ways over the years, right?
21:47Like, Git gives you the ability to merge conflicts as long as you
21:51don't have edits to the same file or the same line in the file, right?
21:57CouchDB and PouchDB, you know, they use what's called last writer wins.
22:01And so it's whichever.
22:03The, you know, all the systems agree that whichever the last one was will
22:06overwrite now, which one was last.
22:09Okay, now we start to get into, like, interesting distributed
22:11systems questions because you can't trust the clocks on the computers.
22:15So, is it the last 1 to the server?
22:18Does there have to be a central server that decides?
22:20Can we get rid of that central server, Johannes?
22:22Can we?
22:23spoiler!
22:24Yes, we can!
22:25Sort of.
22:26Sort of.
22:26We'll come back to that later.
22:28That makes sense.
22:29If the software runs on our devices and if the data is also in our devices,
22:34now the server is kind of demoted in a way no longer necessarily has to
22:40play that absolutely central role.
22:43If something goes wrong, the client can just forget everything and get
22:46everything again from the server.
22:48But that's exactly the step where we want to go beyond to have the work that
22:52we do and the clients to actually trust that and then collaborate with others.
22:58that's right.
22:58And I think the beauty of this model, and we talked earlier about wanting
23:01to make software simpler, right?
23:03A lot of the complexity of the model that we have today comes from having all
23:07these different systems and programming languages and Like different computers
23:12involved in literally every task, right?
23:15Like if I tick a box in a standard web app, I've got my computer, which
23:20then sends a request to the API server, which then does a right to
23:23the database, which then needs to propagate that back to my computer.
23:26So like, like best case theoretical.
23:29We've got three systems involved.
23:31Practically, though, there's also like API gateways and like, you know, request
23:36routers and like all this other stuff is happening along the way, PG proxy
23:40bouncer connection bouncers and all this like extra hidden complexity on top
23:45of the sort of notional three servers.
23:48Whereas if you can move that so that you know, yeah, you tick
23:51the box and it writes it locally.
23:54That's it.
23:54You're done right now.
23:56The problem is, well, how do I let other people know that I took the box?
23:59And so what we're trying to do is set up.
24:00We call this synchronization, right?
24:02So we want to synchronize the state that you have in your local device
24:06to other computers, whether that servers in the cloud or other people
24:09who you're collaborating with.
24:10And so what we're doing is we're kind of changing the relationship where instead
24:14of the server mediating what's happening.
24:18Right.
24:18The client decides what it's doing, and then it lets the server know,
24:21like, Hey, I made this change.
24:22And, you know, we can do that incrementally.
24:25And in fact, what we can do is record all the edits that are happening on your
24:28local machine and then just basically send a log of that to the server.
24:32And then similarly, the server can send you a log of any other
24:36edits that other people have made.
24:38And now you have, in some sense, a much simpler problem, which is
24:40just like, okay, well, I've got all these different sort of changes.
24:43We have a change log, basically.
Local-first Use Cases
24:48So I think there's a lot to unpack there.
24:51So I'd like to understand how can you actually do that?
24:55But before going into that I'm also wondering whether this works well for
25:00some kind of applications, but not maybe for some other applications.
25:04So I can very well imagine how this works for like Apple notes.
25:08I could imagine how this works for like a.
25:11More complex note taking system, maybe all the way to the scale of Notion.
25:17What I'm wondering though, whether there's like some cutoff point where
25:20this is no longer a good approach.
25:23So if I imagine building something like Facebook is that still a good fit?
25:28For local-first software.
25:29Is there kind of like a line in the sand where you say, maybe theoretically still
25:34possible, but absolutely not a good idea.
25:37What is good fit for local-first software and what isn't?
25:41Yeah.
25:41So the strength of local-first software is when users should have
25:45control over their data, right?
25:48And when the devices at the edge need to be able to operate independently,
25:51local first offers a great fit.
25:54Probably not a great fit for an ATM, aTM just deciding when
25:58it's offline to spit out money.
26:00You know, without any way to let other nodes know what it's doing.
26:03When the users have a lot of agency and authorship over the data, local-first
26:07software makes a ton of sense.
26:08If you're trying to manage an external resource, like cash in a bank account
26:12or a meeting room booking or something like that, then it makes less sense.
26:16I think social media is an interesting middle ground.
26:20I think there's a lot of benefits.
26:23to giving users control over their data and an agency over their data.
26:27But there's interesting scale and indexing problems.
26:30So I can imagine, in fact, there are social networks
26:33that are totally local first.
26:35A great example of one is called scuttlebutt
26:37and that's a local first social network.
26:39But you know, it comes with a bunch of architecture trade offs.
26:42It's designed for offline use.
26:44And so that means when you join scuttlebutt, or at least
26:46When I joined Scuttlebutt, you had to download like 10 gigs
26:50Wow
26:51sailboats and like, you know, social media posts that you will never
26:55read just to be able to get sort of bootstrapped into the system.
27:00don't think that's a necessary consequence of local for software.
27:03design that they chose to prioritize, which is like you should be able
27:07to have the whole enchilada.
27:09But yeah, I think again, right, like, I guess another useful thing
27:13to consider is like how often will users engage with the same data?
27:19So, like, Wikipedia, probably not a great local first app in some sense.
27:24Though I'm told Patrick Collison built an offline Wikipedia
27:27as one of his first ventures.
27:29so, you know, I guess there's a use for it.
27:31But, you know, if you only go to an article once and then you read it and
27:35then you never need it again, and that's like a random access pattern and we
27:38can't know what you're going to want to read, then it's probably not a great
27:41fit for local-first software, right?
27:42Like, you're not editing, you're only reading, and you're not
27:44going back to the same data.
27:46On the other hand, like, a recipe book kind of the inverse of that, which
27:51is like, kitchens often have a lot of, like, metal and RF interference.
27:55Transcribed So even though you're fetching data, you might want to
27:57have the recipe, you want to be able back say, what did I make last year?
28:01might be your favorite recipe and you might have a little note
28:04that you've scribbled on it.
28:06absolutely, scribbling in the margins of your recipe books
28:08is a time honored tradition in, like, every household, I think.
28:12And so, right, like, this is where it comes back to that question,
28:16which is, like, is a user taking data for themself and then they can have
28:19authorship over it going forward?
28:21It's probably a good fit.
28:22the more kind of like trust, like control and trust, I think are
28:27related here and the less you can trust all the participants in your
28:32system, the harder it probably will be to build in this local first way.
28:37Since you need to mediate what is typically referred to as auth
28:41and your typical three tier app.
28:44This needs to be thought of probably quite differently when
28:48you build local first apps.
28:50I think this is a bigger conversation for another day.
28:54I believe that local-first software will provide a much higher
29:02standard of privacy, security and authentication than a cloud service can,
29:10I think there's a whole bunch of new best practices that we need to figure out
29:15that might not be fully discovered yet.
29:17yeah, it's definitely an open area of work that I'm not here to
29:20tell you that problem is solved.
29:22I'm here to tell you that, like, the potential, I think, is huge
29:25for this, but the reality is we're a long way from having,
29:28And that's perfect.
29:29that's what I want to help with this podcast give an outlet to discuss
29:33those sort of particular areas, such as what does authentication
29:38look like in a local first way?
29:40And I think something that makes the local first world
29:42already so rich are those like.
29:44Early products that are being built in this way where we can share and learn from
29:49their experiences and their experiments.
29:52So I'm really looking forward to deep dives on those various topics.
29:57tons of people doing great work.
29:58Of course, blue skies out there doing it in more of what you'd call a
30:01federated way, but they're definitely thinking about these problems.
30:03And there's lots of people building products.
Data Syncing
30:07So going a step back again.
30:10When you say synchronization, I'm thinking the Dropbox icon back in the
30:15days when I was using Dropbox a lot.
30:18So that does synchronization.
30:19I think a lot of people are still familiar with that.
30:23Is that kind of like the mental model, how I think about it?
30:26Git also has a notion of.
30:29Synchronizing, if a bit more explicit with push and pull how would I do that
30:35for my data where I think about my data, maybe more in terms of SQL tables.
30:41So how do I translate that very intuitive model of syncing to the
30:47more harsh realities of my app,
30:50This is a great flashlight to shine on like how the world has changed, right?
30:55So if you go and look at Dropbox, what does Dropbox sync?
31:00It syncs files.
31:02What is a git repo on your machine?
31:05A git repo is a bunch of files.
31:08And then git and Dropbox both do something special behind the scenes to get those
31:14files other places or from other places.
31:16Okay.
31:16What is a Google Doc?
31:18You have a Google Doc on your computer, you don't actually have it.
31:22You're just looking at it, right?
31:24And that's why Google can throw away all your data because there have
31:27been too many edits by other people.
31:29Nevermind spending three hours on a flight, rewriting
31:32the stupid blog post, right?
31:33Cause it's, you didn't, I didn't have the doc.
31:36I had a cache.
31:37I had a copy.
31:37I had a view.
31:39And because somebody else had edited it, the actual doc, the place had
31:44changed and mine was now trash.
31:47. Google Docs are places.
31:49And I think this is like a really interesting kind of duality, right?
31:53Because we want both Properties for our documents.
31:56I want to be able to send a link to someone so that they
31:58can edit a document with me.
32:00Right?
32:00That feels like a place.
32:02But I also want to be able to save a copy and email it around or
32:05have it on my computer and look at it and know that it's mine.
32:08Right?
32:08So we want properties of both this kind of like object like
32:13experience that we get from files.
32:15And the place like experience we get from cloud documents.
32:18And I think that's where we get into like a real fundamental
32:20tension between the two.
Existing Local First Products
32:22Do you feel any sort of software that we use on a daily basis is getting
32:27the closest to what you think could be a good resolution of the tension?
32:33I think there are lots of companies that have made incremental progress.
32:40Just to kind of, you know, speak to some exemplars.
32:42I mean, obviously, I think the basic idea behind an app generally, I think it's
32:48kind of problematic, which is it's taking your data and putting it into their app.
32:52So you can't access somewhere, access it anywhere else.
32:55Great example, again, is Apple Notes.
32:58Plain text notes, more or less, but you can't open them in VS code or notepad or
33:04anything else or Dropbox notes, right?
33:06Like, we've managed to turn text notes into closed proprietary formats that are
33:12specific to a single piece of software.
33:14That's sort of a bummer.
33:15You can't sync your Apple Notes with Dropbox.
33:18But okay, I still think though that, like, within this world
33:21of files versus You know, things versus places, files versus links.
33:26Apple notes is like a nice middle ground, right?
33:28Like you can make a note, it always works offline, but you can share
33:31them with other people and see the edits and get updates from people.
33:33So that's really cool.
33:35I think Figma is doing really interesting things with like,
33:38branches and decentralized workflows.
33:41If you ever talk to people who work in a large org that uses Figma, You know,
33:46there's this problem, which is that like when you're working on a design in
33:50the early stages, you don't want other people to look at your work, right?
33:55You're they don't want someone hopping in and going, I don't like that color.
33:58It's like, yeah, I'm not done.
34:01back later, right?
34:02You want that creative privacy where you can explore.
34:04And so people have found all these creative ways to build that in Figma.
34:08And I think there's somewhat cognizant of that organizationally.
34:11So I think that's like a good example of yeah.
34:14Like trying to plumb some of those depths, I think, though, for the most
34:18part, I mean, you know, we could give a bit of credit to Google Docs, right?
34:21Like, yes, it requires a browser extension and like literally half the
34:25time that you at least I get on a flight.
34:27I realized that although I had the browser extension plug installed, it was in
34:31the wrong profile, or I didn't remember to cash the document or whatever, but
34:34you can at least get some semblance of that offline support from Google Docs.
34:40And of course, you know, Git is an exemplar as well in terms of, like,
34:43Git plus GitHub gives you something.
34:46It's clunky.
34:47I'm not here to tell you Git's a great user experience, but, you know, Git
34:51gives you this ability to work kind of in both worlds where you can share
34:54links to files through GitHub, but you can have a full copy of the repository
34:58with its history on your machine.
35:00Right.
35:01Yes.
35:01I think there's a lot to be explored from a user experience perspective,
35:06like patterns that fit the domain of the software patterns that
35:10users intuitively can work with.
35:13But I think in terms of how we actually building that there's probably still
35:17a long way to go and think it's super, super fortunate that there are
35:23companies already pushing forward.
35:25there whether it being Figma or Google with Google Docs or even Notion who I
35:31think is like not super sophisticated in terms of syncing is yet a good
35:36example of like how far you can get while building collaborative software
35:41that is not super fine granular in terms of its syncing resolution.
Conflict Free Replicated Data Type (CRDTs)
35:51And then when you actually get to the implementation, when you're saying
35:55syncing I've already seen the term CRDT quite a bit associated with local first.
36:04Can you briefly touch on what CRDTs are?
36:07We probably don't have enough time to fully go into it.
36:10Maybe you can,
36:11let's give a very high level primer.
36:13I think this is a topic that needs a deeper dive, but
36:16we'll give the quick intro.
36:18So the idea behind the CRDT is that it stands for a conflict
36:22free replicated data type.
36:28data type is pretty easy.
36:30There are lots of different CRDTs.
36:31You can have a set, you can have an array, you can have a counter.
36:35We work on one that's basically like a JSON file.
36:38Right?
36:38So it has sets and arrays and counters and maps and things
36:41and when you say data type, this is now a thing that I'm using to build my app.
36:45So in the same way, if I build a JavaScript app and I would use a map
36:50or set an array, now I would use one of those CRDTs as foundation for my data.
36:57Why are you doing that?
36:58Well, because it's replicated.
37:00What does means?
37:01It means you can take that data type and you can send it to a
37:03bunch of different computers.
37:05All right, cool.
37:06typical JavaScript map can't do that.
37:09Yeah, I mean, you can call toJSON and then post it somewhere.
37:12And that leads us to the last part of the description, which is conflict free.
37:17And that's where the heavy work gets done.
37:20So replicating your data is not that hard as long as you
37:23don't care what happens to it.
37:24You could just, I don't know, open your window and start shouting,
37:27you know, byte codes at the window.
37:29Technically, you're replicating it, I guess, if anybody's listening.
37:32But you know, the conflict free part, that's where it gets tricky.
37:35How do you make sure that if two people both edit the document, One,
37:41that you can merge the changes, but two, that nothing's lost, right?
37:47Like, it's really easy to merge data if you just say, well, I'm going to throw,
37:51always throw away your data and keep mine.
37:54But that's not a very useful strategy, right?
37:56So the whole idea behind CRDTs is lots of different approaches.
38:00Tons of different implementations.
38:02It's a very active area of research and there's lots of great ideas
38:06still unfolding out there right now.
38:08And it's really simple conceptually though, which is like, if two people
38:12edit a thing, you should be able to merge their edits together and both
38:16get the same answer without having to go to recourse to a central authority.
38:21That seems like a very interesting area that I want to learn more about.
38:26When thinking about CRDTs, there are a few technologies that come to mind, whether
38:30that's being Yjs by Kevin Jans, who's been working on this already for many years,
38:35then you all been working on auto merge . So I would love to learn a lot more
38:40about those, but right now we'll shelve it away and say, okay, CRDTs is a great
38:45strategy if you need to implement syncing.
38:48Is that
38:48Yeah, it is, and I think there's a couple of, I'm not a big believer
38:53in silver bullets or kind of like magical thinking with technology.
38:56There's a set of tradeoffs here, right?
38:58Like, what happens when you have a conflict in git?
39:01Well, you're busting open your text editor and looking for a bunch of less than signs
39:06in a row and hoping you found them all before you push the stupid commit again.
39:10You know, similarly, CRDTs don't generally have that problem, but
39:15you can still have conflicts, right?
39:16Like if you and I both edit the title of a document, You know, the CRDT isn't going
39:21to be able to know which one is right.
39:24So there's interesting, just like, challenges here that go beyond,
39:28abstract computer science problems.
39:31This is a really fundamental problem.
39:33The computer can't know what the title should be, right?
39:36And in the central cloud model, what happens is, like, one of us will find
39:40out when we go to write that we lost.
39:42And so you know the problem happens right away.
39:45And so what makes CRDTs interesting from like a user experience design problem is
39:50that you can have conflicts discovered a long way away from where they're
39:56created, either in space or in time.
39:59And so that actually has serious user experience consequences.
40:03This can also happen to you and get right.
40:05Like all of us developers listening here know what it's like.
40:09To, like, try and merge a long running branch and realize that
40:11somebody else has, like, completely rewritten a file you edited and
40:14you're like, oh no, what do I do?
40:17Do I just throw my entire week of working away and start over?
40:20Or do I, like,
40:21There's no free lunch here.
40:22So it's semantically diverges a lot or changes a lot, then the
40:26user still needs to help out.
40:28And similarly, the CRDT can't actually recognize all the conflicts, right?
40:32The technical conflicts we can catch.
40:36But again, if you sort of think about software, Git is kind of
40:40like a bad CRDT in some ways.
40:42Even if we edit different files, that doesn't mean the tests will
40:46pass or the software will be useful.
40:48I may have renamed a function that you were using.
40:51Both can commit cleanly, but then when we try and run the tests, it
40:54won't even parse and it crashes.
40:56We can think about all those things, and I think that's the, design side of the
41:00problem, and then, there's lots of neat stuff to talk about on the implementation
41:03side, but we can on from that for now.
41:05We'll back to those later.
41:06Yeah, so going beyond syncing and CRDTs, obviously that's a very
41:11important aspect, syncing in general for building local-first software,
41:15as it's one of the pillars, like the software should work on my device,
41:20and we should be able to collaborate.
41:22So if we come back to the.
The Challenges of Building Local First Apps
41:25Which sort of problems does a developer who's typically rather
41:29building that three tier web app, what sort of new problems does the
41:34developer now need to think about or maybe think about differently aside
41:38from syncing if you want to build your next app in a local first way.
41:42The biggest one is architectural.
41:44When you grow up in this environment of you know, the
41:48database is the source of truth.
41:49And then things sort of roll out, you know, now you have different problems.
41:53Like, how do I make sure that I have the data, you know, when my user is offline?
41:57Can I make sure that I've fetched everything they need so that
42:00they know before they go offline?
42:02Like, yes, you're up to date.
42:04Similarly, you know, if you can be partially online, right?
42:09Like, maybe you're online and you can talk to the sync server.
42:12But somebody else can't.
42:15So, you know, how can you know that you're seeing what other people
42:18are seeing that you're seeing the things somebody else has sent, right?
42:21I like to give examples from the field when we talk about this stuff.
42:24Facebook Messenger has like a really simple version of this problem, you
42:29can still reply to a message when you're offline with your phone, but
42:31you don't get the little checkbox that says, Yep, that's been sent.
42:35And it's not until somebody else takes the phone out and looks at it
42:38and the little, you know, miniature avatar icon drops below the message
42:42that you know that they've seen it.
42:45What does that mean for source code?
42:49Or like a text document.
42:51I mean, we can't just move the avatar to the bottom of the page, right?
42:54So there's like really interesting challenges here around kind of,
42:57thinking a little differently about how you build your app, right?
43:01And similarly, just because you have the data locally or could
43:04have the data locally doesn't mean that it's going to load quickly.
43:08If you store everything as like a long log of edits in an indexed DB,
43:12it could take you longer to load than if you were fetching it from the web.
43:15So we need to be very thoughtful about these things.
43:17But the result of Using this approach has one really killer benefit that I
43:23just still delight in every day, which is that, you know, there's the old thing
43:28about, you know, it ran on my machine.
43:30Well, we'll ship your machine.
43:31And that's how Docker was born, right?
43:35Like, it's sort of like that, though, with local-first software, like, if
The Benefits of Local First for Developers
43:41You have the whole thing.
43:43So you're done.
43:44Right?
43:45Like, as long as you have a sync server somewhere, you're finished.
43:48And actually, maybe this is a good moment.
43:49Can I talk a little bit about peer to peer for a sec?
43:51Please.
43:52Yes.
43:53So, I think one misconception I hear from people is like, Oh, local first.
43:57That means you're anti cloud and you don't like servers.
44:01Not at all.
44:02Like, peer to peer technologies are interesting and there have been You
44:09know, use cases where they were really vital to achieving users aims, like, you
44:14know, when you want to download an entire album full of Linux ISOs, you know, then
44:21BitTorrent is, like, really good, and you can see why having central servers
44:24would be a problem for that, but in the case of most software, you know, one,
44:31peer to peer systems actually don't work very well in a lot of network conditions.
44:35You can, you know, you can run BitTorrent on your home network with a little
44:39bit of work or a VPN, but at the coffee shop or places like that, those peer
44:44to peer systems really struggle because the network tends to be configured in
44:46a way that doesn't allow them to work.
44:48But more than that, right?
44:50Like, if you and I are collaborating on a peer to peer system, right?
44:54Like, I'm on a laptop most of the time.
44:58I don't want my work to disappear completely just because I close the lid,
45:03We want servers.
45:04We want them
45:07mean, they literally should serve us.
45:10They right, not us serving them, right?
45:12We're trying to make the server serve us rather than vice versa.
45:17Love it.
45:18That's great.
45:19And so, yeah, the big difference here in this model is that you still want
45:23servers, but their responsibility is just.
45:26to host data and distribute it.
45:29And so, you know, yeah, you want it.
45:31It needs to be online for things to be successful because peer to
45:34peer systems don't really work.
45:36I have spent many years trying to like adopt peer to peer architecture
45:41before kind of finally giving in and embracing this perspective.
45:45And then even if they do work, they don't solve the problem, which is
45:47that in fact, you want your data to be online even when you're not
45:51Right.
45:52You know,
45:52there's an interesting role it can play.
45:54I know a few folks who haven't given up yet on peer to peer.
45:59And maybe in the cases where it does work, it can be an optional part of the process.
46:03Of a technology stack, let's say you're in a smart home setting where you
46:07can assumptions about software works.
46:11Well, peer to peer software is great and can work really well, but it, I have not
46:19seen a design or implementation of peer to peer systems that are like really
46:26robust in all network environments and where you don't just actually have a few
46:33centralized servers hiding off stage.
46:36While you're saying peer to peer, but actually, you know, 20 percent of
46:39traffic is going to some server like a great example here is WebRTC, right?
46:44If you're familiar with WebRTC, people think of it as like this
46:46peer to peer system built into the browser, but it's not for two reasons.
46:50First.
46:51While it is true that WebRTC allows two computers to talk directly to each other,
46:56in order for them to find out about each other to have that conversation,
47:00you need a server to introduce them.
47:03The second is that, and opinions vary about the prevalence of this, but I have
47:09seen credible reports that are sort of on the order of 20 percent of WebRTC
47:14connections, cannot be made peer to peer.
47:18And are instead sent through what's called a turn server, which is just relay
47:22that just bounces the traffic for you.
47:25And like, that's not to say WebRTC is bad.
47:28I mean, it's super useful.
47:29We use it every day in our, like, video conferencing lives.
47:34But that, it's not really like peer to peer.
47:37It's like client server with a peer to peer fast path when it's available.
47:42I think there's lots to be explored as well.
47:44That's more like on the networking side.
47:47And I think depending on for some use cases, like . Let's say you
47:51build software for a warehouse or maybe a cruise ship or something.
47:55I think there, there might be real use cases, and I'm really
47:58you have control...
47:59are exploring it.
48:01Yeah, if you have control over the network, you totally can and actually
48:05our friends at ditto where former lab collaborator Ray McKelvey works have
48:10built a really great business building local-first software using a different
48:14kind of CRDT than the one we use for airplanes and like fast food restaurants.
48:20They have like real scaled out production, industrial deployments
48:24of this technology in the field.
The Seven Ideals of Local First
48:29So if you Google Local First right now, you might come across the blog post
48:34and essay that you've been a co author of where you first introduce the ideas
48:39of Local First, and as part of it, you have those seven ideals of Local First.
48:45So I think this would be enough to, fill an entire episode on to really
48:48do each ideal justice, but maybe you can just Summarize those seven
48:54ideals and what they mean for you.
48:57Yeah.
48:57Well, we've talked a bunch already about how building local-first
49:01software is good for developers, right?
49:02Like, your distribution costs are better.
49:05Your on call experience is better.
49:06Your software is less complicated, hopefully.
49:09But the original motivation behind developing the ideas of local first was
49:13really focused on kind of user experience and not in the sort of what corner
49:18radius should the buttons be or what shade of green for the call to action, but like
49:23the real core experience that the user has a person who's consuming your software.
49:28And so we identified with Martin Kleppmann and Adam Wiggins and
49:32Mark McGranagan seven ideals.
49:35That we think local-first software should aspire to the first one is no
49:39spinners right next frame or your money back like you should never go to open
49:47something and just see like a spinner because your network connection is
49:50bad or whatever else like that's just it's inexcusable we have the fastest
49:54computers in history with the biggest drives ever in the most bandwidth like
49:58it's just ludicrous that our own personal data would not be available to us.
50:03Number two your work is not trapped on one device.
50:06No one wants to be in a world where like you dropped your phone, over a cliff
50:11or off a ledge or whatever and suddenly your data is all gone with it, right?
50:14You left your laptop on the train and now you've lost your life's work.
50:18No, we don't want that.
50:19So your work should be able to synchronize across all your devices
50:21and be available on every device.
50:24Simultaneously with this idea of not having spinners, right?
50:26The idea is we want all seven of these things to be true together.
50:30Number three is that the network is optional.
50:32As we've discussed, like, you often find yourself in places where the network
50:36either is not available because of technical reasons, or just because,
50:41you know, you want to be offline.
50:43Right.
50:43There's something to be said for making that choice, turning off
50:45the Wi Fi signal, staying focused, and still being able to work.
50:49We still insist on seamless collaboration with your colleagues.
50:53That's number four, right?
50:56It's, if it's not collaborative, it's local only, not local first.
51:00Another one that I think is really important is this notion,
51:04number five, the long now.
51:07I can get on a single micro SD card.
51:13Like, every game that was ever released for Super Nintendo.
51:16And those will work a hundred years from now.
51:20Like, how much of the software that you've built in your career
51:23will work a hundred years from now?
51:25How much of it will still be around ten years from now?
51:28For many of us, so much of our work just evaporates.
51:31And it's not just for the vanity of wanting to see your work survive.
51:37When you're talking to people, writing books, or...
51:41You know, who have built their careers around like a long term
51:44project, you know, it could take 10 years to do the research for a book.
51:48How many cloud services that were around 10 years ago are still around?
51:52Or how many of that are around today will be around in 10 years.
51:55You know, you can't control who's going to get acquired or shut down.
51:59So I think that's a big part of it.
52:01Number six is security and privacy by default.
52:04You know, why is the default that when I use software, the product managers at
52:09the company or the support people can just read my data whenever they want?
52:13I mean, they don't.
52:15I hope they don't.
52:16They most...
52:17Places say they won't,
52:19Right.
52:20but you know, they, they could, right.
52:23That's pretty messed up.
52:25And then number seven, ultimately, you know, we want the users to
52:29retain ownership and control.
52:31Right.
52:32And that's in some sense, kind of a distillation and restating of
52:35some of the other points, but like.
52:38You know, if someone can take it away from you, it's not yours.
52:41We want you to be able to have the software to make decisions
52:44about when you update it.
52:45If you update it, who you share your data with, when you share
52:49your data with them and to put those tools into the user's hands.
52:52We want them to have that feeling.
52:55It's a genuine fact of having the thing and having agency and ownership over.
53:03those are the seven ideals.
53:04I think it dovetails with a lot of the developer benefits, right?
53:08Like, sort of the same underlying decisions that make things
53:11easier for developers also can improve things for users.
53:15But those are the seven ideals.
53:17I would say from a product perspective and from the end user perspective, I think
53:21there's nothing controversial about it.
53:23I think like almost all developers who build apps for the cloud today would
53:28probably still say, yes, those would be great characteristics for our apps.
53:33It's just hard to build apps like in a cloud first way that have
53:38all of those characteristics.
53:39So I think this is why we need a bit of a different model, how to build software.
53:44And, those from a user perspective are amazing.
53:47And from a developer perspective, everything becoming
53:50simpler sounds incredible.
How ready is Local First?
53:53Zooming out a bit, I love all the ideas of Local First, and so this is not the
53:59topic of today's show like, I'm also, for the last two years, working on a
54:05Local First app myself, but, Pretending I'd be two years earlier and just trying
54:11to make sense of what is local first?
54:14Is that for me?
54:15How do I go about it?
54:17Let's say I'm at the beginning of my journey and I'd be asking you as
54:21a fellow builder for advice, like, Hey, Is local first ready for me
54:25to build my next app with it today?
54:28And if so what does my typical path look like?
54:31Is there like a Ruby on Rails for it already?
54:34Is this a good fit for me?
54:36Do I rather have to be like an adventurous explorer?
54:39Or can I already go into this with a mindset of what
54:42is the most popular stack?
54:44And that's also what I got to adopt here,
54:47So we're recording this in 2023.
54:49And today I would say building a local first web app is likely
54:55to require inventing something at some point along the way.
55:00Like this is not like just pushing stuff to Netlify and following
55:04posts off stack overflow.
55:05This is a new thing and you're going to have to figure some things out
55:09and I hope someday that this is as easy as falling off a log, right?
55:13Like at Heroku, we used to always say make the right thing the easy thing and
55:18make the easy thing the right thing.
55:20And so if we can do that, if we can get there together as a community, then, you
55:24know, Hopefully when people find this podcast a few years from now, they're
55:28like, wow, we've come a long way.
55:30But today I think, you know, what I would recommend is like,
55:33you can build a react app.
55:36You can use a CRDT like, Yjs or auto merge are probably the most
55:42mainstream choices right now.
55:44But there are some technologies that I can
55:46Oh yeah.
55:47And there's a Vulkan.
55:48io.
55:49There's ElectricSQL.
55:50There's like a ton of people coming out and trying things.
55:54And depending on what looks good to you you should try one and see how far you
55:59So there's basically an ecosystem of tooling already for those use cases.
56:03there's always a bit of like, it depends.
56:05So one might better for your use case than another.
56:08But think what saying is like, I need to invent maybe something along the way.
56:13Maybe I need to do end to end encryption, or maybe I need to do
56:17permission control in a certain way.
56:20Where none of the available tools yet do exactly what I need.
56:23And this is where I need to get my own hands dirty,
56:26Yeah, you'll probably need to figure out how to deal with
56:30some of those offline use cases.
56:32You're going to have to decide who you want to give this data to.
56:34I think that's one that everybody kind of wants different
56:37things when we talk to people.
56:39And I think eventually some standards will emerge.
56:41There'll be probably two or three patterns.
56:42Most people adopt, but at the moment, because everybody's asking
56:45for different things, people are all making different things.
56:49But I think that's a that sort of ACLs and access permissions is an area you should
56:53expect to spend some time thinking about.
56:55But like, I gotta say, it feels amazing to just stand up an app.
57:02In a short matter of time pointed at some kind of a sync server and then just be,
57:08you know, working on it with other people.
57:10And I hope everybody will take a take the time to at least have that
57:15experience to try building something small, whether it's with automerge or Y.
57:18J.
57:18S.
57:18or ElectricSQL or anything else.
57:21And that experience, it's just.
57:25It's so eye opening for me as somebody who spent all these years building these big
57:30complicated web apps for the cloud to just say actually, it can be pretty simple.
57:37And when it is simple, it feels great.
57:40You feel like you have like, like wings.
57:43You're just able to move so fast, you're able to get so much done.
57:46And it's like someone's taking the brick off the pencil.
57:48You just, like, you don't even realize how much hassle you're putting up
57:52with every day until it goes away.
57:54So maybe after someone's listening to this, maybe on your next weekend,
57:58give it a little try, like build a app you want it to build all along, maybe
58:03React app, like throw in automerge, throw in Yjs, see how far you'll get.
58:09And I think the promise is really that users, your end users
58:13will get much better software.
58:15And for you as an app developer, your life becomes a lot simpler.
58:19You can take off that, that big brick and just enjoy drawing
58:23and writing with your pencil.
58:24That, that sounds super exciting.
58:26And like having seen the ecosystem now for the last two years evolve, I think we
58:31have already come quite, quite a long way.
58:34Now there is a real ecosystem and you don't need to reinvent
58:37everything by yourself.
58:39If someone's like looking for some references or some inspiration, can you
58:44point to like any sort of software that is already local first, or I guess like
58:50local first is a bit of a spectrum.
58:52Some apps might be more local first than others, but is there like some
58:57software that people already use today that, that is directionally local first
59:04that you can orient yourself around?
59:06Just looking at the core set of apps on your phone is like a nice starting point.
59:11For web apps, there's things going back all the way to like ether pad.
59:15Right.
59:15Where you have offline collaboration.
59:17And then if you want to try and build your own experience
59:21automerge.org/docs/quickstart.
59:24You know, you can get up and, you know, experience that feeling of building
59:27local-first software for yourself.
59:29I think most apps that we from a user perspective, see as working fast or
59:35that you also trust to use them while you're traveling on a plane, on a train.
59:40I think they are all probably, even though they don't have a local first
59:44label yet on the software box, they are probably already more local first.
59:49And yeah, this is, I think, whoever's trying to build something
59:54next if you get to experience that for yourself, I think that's the
59:57easiest way to convince yourself.
59:59I'm thinking of some software that I think that I use on a daily basis,
1:00:04whether like, whether it's like, something like, Linear or something
1:00:08like superhuman, those sort of apps, while I'm not sure whether they were
1:00:12from the inception, meant to be local first, I think they've directionally
1:00:17evolved in a way that is more and more local first, there's probably still like
1:00:21servers heavily involved, but in terms of that client is getting a lot smarter.
1:00:27I think that's already an important step in that direction.
1:00:30And that's certainly why those apps feel great.
1:00:34Amen.
Open Research Questions
1:00:36So maybe before wrapping up, what are some topics within Local First that are
1:00:41very top of mind for you right now that you spend a lot of time thinking about
1:00:46and where you hope that we'll be a lot further along in one or two years time?
1:00:51Ooh.
1:00:52I think for me, the biggest design questions right now are around
1:00:57versioning and version control.
1:01:00One of the things we've found as we've studied local-first software and talked
1:01:03to writers and creators is that, um, this idea of the cloud as like, Sort
1:01:10of the panopticon where anyone can watch you work is like really deeply
1:01:14uncomfortable to a lot of people and so exploring these ideas of creative
1:01:18privacy and the ability to work the way software developers work, which is you
1:01:22work private until you're ready to share.
1:01:24I'm really eager to explore those ideas and bring those.
1:01:28To the rest of the world, your Google Docs, you should be able
1:01:30to work in that same offline way.
1:01:33So I think that whole...
1:01:34Question about like bringing together version control in new ways is a big one.
1:01:39Authentication, authorization, access control, sharing
1:01:44permissions, that kind of stuff.
1:01:45I think it's actually quite closely related.
1:01:47I think that's an area we're going to see a lot of progress around.
1:01:50I think the app models are going to continue to evolve, right?
1:01:54Like.
1:01:55Auto merge is document based, electric SQL is database based and Riffle is SQLite.
1:02:01Both are actually right.
1:02:02I don't think it's one or the other, but I think exploring kind of that relationship
1:02:06there is going to be a big area.
1:02:08I think there's a lot of stuff around security and encryption,
1:02:11end to end encryption that's going to unfold and with it.
1:02:14What's that what's that quote?
1:02:16I think it was Leah Kistner, which is , cryptography is a
1:02:19technique for turning many problems into key management problems.
1:02:23so I think we're going to see a lot of that over the next few years.
1:02:26I think that's just scraping the surface.
1:02:28And I think all of this though will also intersect with like
1:02:31application development models.
1:02:33Just thinking about other kind of intersecting things, another big part of
1:02:37where we're still relatively immature is deciding what to sync and when to sync it,
1:02:43There's this motto we have at Ink and Switch, which is like, results on
1:02:46the next frame are your money back.
1:02:48We want everything you do to run at a hundred hertz, none of this like a
1:02:52hundred millisecond round time to the server, and then you get to paint.
1:02:55No, like when you interact with the system, you should see the response
1:02:59the next time your screen refreshes.
1:03:01So that means on a hundred hertz screen, you got 10 milliseconds to do everything.
1:03:04And so if the server is 50 milliseconds away is too late,
1:03:08you can't ask the server, right?
1:03:10If you want to have things by the next data, that means you
1:03:12need to out there already get the data before the user even asks
1:03:15That has big implications on your data architecture on like performance,
1:03:20So many Yeah.
1:03:22Things need to be in memory.
1:03:23How do you manage memory use?
1:03:24What about like low bandwidth connections?
1:03:27So yeah, I think that whole area of like sync optimization and background
1:03:31sync and so on is really important.
1:03:33And it's an area that have thought about, but we haven't seen a ton of
1:03:37progress on implementations yet, because until you actually have enough users
1:03:41and enough usage to really drive that,
1:03:44Really interesting topics to dive deeper on in future episodes as well.
1:03:48There is one topic that also stands out to me that I think we're not there
1:03:52yet which is cross app collaboration or cross app interoperability.
1:03:57So as we're.
1:03:59Steering in the direction where we have like more and more AI in our
1:04:03lives and all of those AIs so far are looking also that they run on like
1:04:08somewhere's remote server, et cetera.
1:04:10And now we're going to feed more and more of our data into those I
1:04:15would love to see a world where like the AIs can also similar, like the.
1:04:20Like the programs that we're building also run on our devices.
1:04:23Now we need to make our data available to those.
1:04:27Have those programs and those AIs collaborate between each other.
1:04:31So in the same ways, like all the stuff I have in my own head.
1:04:36From conversations with you, from conversations with others, my mind is
1:04:40like able to, all of those different interactions I have are almost like
1:04:45little apps and they naturally can collaborate yet my contacts app that runs
1:04:50on my machine and my email app, unless they're from Apple and does integrate it,
1:04:56otherwise they can't like, what does that integration story between apps, et cetera?
1:05:01What does that look like?
1:05:02And I think the foundation for that is.
1:05:05The data needs to be on the device.
1:05:07And now we need to build something on top that makes it easier to interoperate.
1:05:13That data is shared across apps.
1:05:15That's a whole topic that I'm also really excited about.
1:05:18So we call that malleable software at Income Switch, and it's a very
1:05:22active area of research for us.
1:05:24We've just published a paper called Embark Today as of this recording about that
1:05:29https://inkandswitch.com/embark where we talk exactly about some of these problems.
1:05:35One really cool thing about auto merges design that I haven't seen
1:05:39in other systems is that the grain of the system means that any apps
1:05:46which share a backup sync server.
1:05:49can load each other's data as long as they know how to interpret it.
1:05:53And, you know, it's a little bit like sharing a SQL database, but without
1:05:56all the problems of sharing a SQL database, because the database is not
1:05:59like the shared resource that you have to worry about people hammering, right?
1:06:02you can just put data in a sync server and then anything can query it out.
1:06:06And so, I got to say, it's awesome.
1:06:09I use it all the time.
1:06:10When I break something, I can open it up in another app and fix it.
1:06:13You know, I have a whole suite of little apps that are local first
1:06:16that collaborate on our data now.
1:06:17And yeah, well, we'll say more about that sometime in the future
Outro
1:06:22Peter, thank you so much for sharing all of that about Local First with us.
1:06:28I think this is the beginning of a much, much longer journey where I want
1:06:32to hear a lot more different voices, like people building apps with Local
1:06:36First, people building tools to help other developers build Local First apps.
1:06:42All the things we've just barely touched the surface here today.
1:06:46So thank you so much for coming on the show and I'm looking forward
1:06:48to having you back hopefully soon.
1:06:50Yeah.
1:06:51It was a lot of fun.
1:06:52Take
1:06:52Awesome
1:06:53Thank you for listening to the Local First FM podcast.
1:06:56If you've enjoyed this episode and haven't done so already, please subscribe and
1:07:00leave a review wherever you're listening.
1:07:03Please also consider telling your friends about it, if you think they
1:07:06could be interested in Local First.
1:07:08Thank you again to Expo and Crab Nebula for supporting this podcast.
1:07:12See you next time.