localfirst.fm
All episodes
February 14, 2024

#3 – Geoffrey Litt: Malleable software, local state management & Riffle

#3 – Geoffrey Litt: Malleable software, local state management & Riffle
Sponsored byExpoCrabNebula
Show notes

Transcript

0:00:00 Introduction
0:00:00 isn't it really complicated to build web apps?
0:00:02 What if it was just way simpler to build an awesome app that was fast, that was
0:00:06 really responsive, that stored all the data people care about, was easier for
0:00:10 you to develop faster, simpler, less ops maintenance stuff The data ownership
0:00:14 stuff just falls out as a consequence.
0:00:16 Could we leverage the power of databases and the power of local-first
0:00:21 architecture to cut through a lot of the complexity of building
0:00:24 modern web apps and make it simpler.
0:00:26 Welcome to the localfirst.fm podcast.
0:00:29 I'm your host, Johannes Schickling, and I'm a web developer, a
0:00:32 startup founder, And love the craft of software engineering.
0:00:35 For the past few years, I've been on a journey to build a modern, high quality
0:00:39 music app using web technologies.
0:00:41 And in doing so, I've been falling down the rabbit hole of local-first software.
0:00:46 This podcast is your invitation to join me on that journey.
0:00:50 In this episode, I'm speaking to Geoffrey Litt, who's currently
0:00:53 a researcher at Ink and Switch.
0:00:55 Geoffrey has worked on many interesting research projects such
0:00:58 as Riffle, Cambria, and Embark.
0:01:01 In our conversation today, we're talking about the ideas and goals
0:01:04 that motivated Riffle, Which include expressing an app as a query, fast
0:01:09 synchronous reactivity, and unifying UI state and app data in a single system.
0:01:15 Before getting started, also a big thank you to Expo and Crab
0:01:18 Nebula for supporting this podcast.
0:01:21 And now my interview with Geoffrey.
0:01:23 Welcome, Geoffrey, to the show.
0:01:25 So excited to have you.
0:01:26 Thanks.
0:01:26 It's great to be here.
0:01:28 Malleable software
0:01:28 I mean, you and I have been working together now for a few years, But let's
0:01:32 rewind time a little bit and maybe start where you and I have first crossed paths.
0:01:38 This was, I think the end of 2020 and you've been just wrapping
0:01:42 up your first project at Ink and Switch, which was about Cambria.
0:01:46 So would you mind, motivating what the project was about?
0:01:51 Yeah.
0:01:51 Absolutely.
0:01:51 So, I'm a researcher that works on a topic called malleable software, where the idea
0:01:57 is, can we let everybody customize their software tools to meet their own personal
0:02:02 needs rather than everybody changing the way they work to match standardized
0:02:07 software tools that other people develop.
0:02:09 Right?
0:02:09 turns out one of the core problems in this space is wrangling data schemas.
0:02:15 So imagine, you know, you're using your favorite to do app, I'm using
0:02:18 my favorite to do app, and we decide we wanna work on a project together.
0:02:22 What are the options?
0:02:23 We either have to pick the same to do app, or if we wanna each keep using
0:02:27 our own favorite to do app, those apps need to have some way of talking to one
0:02:30 another and sharing a data representation.
0:02:33 Right?
0:02:34 it turns out that this is a pretty gnarly problem because those data
0:02:38 representations our 2 apps wanna use might be different in subtle ways.
0:02:42 You know?
0:02:43 Perhaps your app has subtasks and mine doesn't or something like that.
0:02:47 And so the goal of Cambria was to explore this problem and try to figure
0:02:51 out, could we find a way to enable collaboration across diverse tools,
0:02:55 while also not requiring all of our tools to share exactly the same data
0:02:59 schema and all agree on everything.
0:03:00 And It turns out that this is actually sort of a pervasive problem
0:03:05 across local-first software in a bunch of different contexts.
0:03:08 This problem pops up in many shapes and forms.
0:03:11 Even the simple case of, like, having a centralized application written by
0:03:14 one developer in a local-first context schema migrations And upgrades often
0:03:19 become a lot harder than they are in sort of a server based environment.
0:03:23 So, the Cambria project again was about sort of First, just recognizing that
0:03:28 this problem exists everywhere and that it's really holds back progress
0:03:32 in terms of interoperability and schema management and local-first steps.
0:03:37 And then we sort of prototyped a solution to it, which was you can think of it
0:03:41 sort of as a live data translation system where as different tools are
0:03:45 working on shared data, there's a layer underneath the individual tools that
0:03:49 helps translate data formats between all the various tools in such a way that
0:03:53 each app can feel like it's just working with the data representation it wants.
0:03:57 But, actually, under the hood where we have some sort of translation layer
0:04:01 that's, you know, sort of serving as the glue that lets different
0:04:05 versions of the same app collaborate or lets even different apps entirely
0:04:08 collaborate with one another.
0:04:10 Yeah.
0:04:11 I think that's a brilliant motivation.
0:04:13 I remember, like, having read the essay the first time was super well
0:04:18 illustrated and super well motivated.
0:04:21 It sort of almost provoked me into reimplementing a version
0:04:26 of Cambria myself back then.
0:04:28 This is how much it's it kinda spoke to me since I remember that the
0:04:31 first version of Cambria was an sort of like a DSL expressed in YAML.
0:04:36 I am more someone who's working mostly in TypeScript, so I remember reimplementing
0:04:41 a small version in TypeScript.
0:04:43 So, yeah, I love that problem statement.
0:04:46 And so I think it's also interesting how it fits into this broader
0:04:50 umbrella of malleable software.
0:04:52 and turns out, Looking back over the last few years, this is not
0:04:56 the only project you've done in the realm of malleable software.
0:05:00 So which other problem spaces have you run into while pursuing
0:05:04 this goal of malleable software.
0:05:07 Yeah.
0:05:07 I think it's a fascinating space.
0:05:09 You know?
0:05:10 The way I got into it was originally it was actually I was working at a
0:05:13 start up building normal SaaS software.
0:05:16 And you know, we were doing what SaaS companies do, which is that you
0:05:19 have 30 people in a room somewhere, designing interfaces that thousands
0:05:24 and thousands of people, thousands of miles away are supposed to use.
0:05:27 Right?
0:05:28 We were, in our case, designing education software for schools.
0:05:31 I just remember talking to all these teachers and principals and all these
0:05:34 different schools, and they had very specific requests often as customers do.
0:05:39 And, of course in SaaS, typically, the answer is no.
0:05:42 You have to be disciplined and say, I can't change the wording
0:05:46 of that button just for you.
0:05:48 I'm not gonna add a setting for that.
0:05:50 But I started to find this very frustrating and wanted to
0:05:53 explore what would it feel like if software worked differently.
0:05:57 And, you know, I think there's an interesting connection to local-first
0:06:00 because I remember coming across the original Ink & Switch essay
0:06:04 about local-first when I was starting to think about this stuff.
0:06:07 And I think they said briefly in the essay something about how It
0:06:12 seems like an intriguing foundation if you have data locally and
0:06:15 maybe even code locally available.
0:06:17 That gives you a little bit more access and control over your software than
0:06:21 if You have a cloud SaaS architecture.
0:06:23 And that sentence really sort of sparked a bunch of thoughts in my brain that
0:06:28 have led you know, I think a lot of my work, the way I see it is sort of at
0:06:31 the intersection of how do we use the local-first architecture to enable more
0:06:36 malleable software for end users is a theme that I care a lot So Cambria was
0:06:39 definitely one project in that line.
0:06:41 I've worked on a couple other projects in this space.
0:06:46 one project I worked on early in my grad school research career was a little
0:06:51 browser extension called Wildcard.
0:06:53 And the idea was what if you could access the data underneath any web
0:06:58 page in a spreadsheet uh, but it's not only a read only spreadsheet.
0:07:01 You could even write to the spreadsheet and mod it yourself.
0:07:04 So a very accessible way for normal people to make their own browser
0:07:08 extensions without programming.
0:07:10 I love that.
0:07:11 And I think as a fun fact, this might have actually been the very first
0:07:16 point where we've been put in touch.
0:07:18 Because at that time, I was also working on my little own Chrome
0:07:22 extension, which was also kinda in a direction of malleable software.
0:07:26 This was before I was even aware of the term and, like, that broader field.
0:07:31 I was just motivated by my own frustration that I could that the websites didn't
0:07:37 quite work the way how I intended.
0:07:39 So it just feels like we've been on similar paths there, And what
0:07:43 you've been building with Wildcard was quite a step ahead of what I've
0:07:48 been coming up with at that point.
0:07:50 But yeah,
0:07:51 Well, yeah, I, you know, I remember one thing about that that I think
0:07:54 maybe is a preview of the way we've collaborated since then is that mine
0:07:57 was a more speculative prototype of a interface design, but it wasn't
0:08:02 particularly solidly implemented.
0:08:04 And I remember you had this really, fancy developer experience around
0:08:08 customizing your extensions and stuff that I got inspired by.
0:08:11 And so I think There's a fun interplay there we can get into
0:08:15 later about kind of more speculative research versus engineering
0:08:18 more solid versions of the idea.
0:08:21 Yeah.
0:08:21 So you've been working on on Wildcard.
0:08:24 Which are the results that come out of your your overall research in that field?
0:08:30 From Wildcard to Riffle
0:08:30 Yeah.
0:08:30 So, you know, throughout the years, I've had the chance to work on
0:08:34 A bunch of projects with mostly with really awesome collaborators.
0:08:38 one of the more recent projects which I we actually worked on together, was
0:08:42 the the last project I worked on in grad school, which is called Riffle.
0:08:46 And that was a project that you and I worked on together with, Nicholas Schiefer
0:08:51 and Daniel Jackson, my PhD advisor at MIT.
0:08:54 And sort of the, I guess, the origin story of Riffle, is that Nicholas and
0:08:59 I were having a bunch of conversations that revolved around sort of this
0:09:02 idea of having a personal data store.
0:09:05 So, one way I think about cloud versus local-first is that in
0:09:10 local-first software, one of the ideals is that you own your data.
0:09:13 these valuable thoughts that I'm having and artifacts that I'm producing in the
0:09:16 digital world, they should feel like mine, just like a notebook I can carry around
0:09:21 and, you know, have with me forever.
0:09:23 Right?
0:09:23 And There's this question of great.
0:09:26 If you own your data sure, you get longevity and control, but
0:09:30 what about this ability to take arbitrary tools that I wanna use and
0:09:35 connect them to that personal data.
0:09:37 This is something that's often really hard in SaaS because the data's locked away.
0:09:40 And maybe there's the APIs you want and maybe there's not.
0:09:43 And maybe there's the partnership you want between the 2 tools
0:09:46 you use, but maybe not.
0:09:47 And we were talking about, you know, how would it feel if you had a local-first
0:09:50 data store with all your stuff in it And you could connect whatever app
0:09:54 or tool you wanted to that existing data store that has all your stuff.
0:09:57 Right?
0:09:58 Um, what shape would that data store take?
0:10:00 How would it work?
0:10:01 How would you develop against it?
0:10:03 And one hypothesis we wanted to explore, particularly because of Nicholas's
0:10:07 background, which is in databases, we wanted to sort of ask this question of
0:10:11 what if you had a really powerful database that was designed for this purpose?
0:10:15 Maybe it could be based on relational databases.
0:10:19 And maybe the experience of building an app in that foundation could end up
0:10:24 feeling really different from traditional software development in a bunch of ways.
0:10:27 So that's kind of how we got into the Riffle work.
0:10:31 That that's incredible.
0:10:32 And if I remember correctly, This was not the first time that Nicholas
0:10:37 was also working on on this field.
0:10:39 He didn't just work on on databases, but I think he was at at Apple
0:10:43 previously and was implementing similar systems, I think, related to
0:10:48 to iCloud and how syncing works there.
0:10:51 So yeah.
0:10:52 I mean, those are some pretty big ideals,
0:10:54 That is not a very common thing in the apps that that I'm using.
0:10:58 It's, if, God forbid, something like Notion would shut down.
0:11:01 Like, what what would even happen to to to my data?
0:11:05 So I would love to see that flip of, like, Everything that I'm creatively
0:11:10 producing is by default mine and turn around the relationship.
0:11:15 But Uh, it's not even that you wanted to, from a idealistic perspective,
0:11:20 push for that, but I think you took a step back and say, okay.
0:11:24 What is even the technical foundation for that?
0:11:27 What would a prototype of that look like?
0:11:31 Data ownership: SaaS vs Local-first software
0:11:31 Yeah.
0:11:32 So one way that I think about this is exactly the way you put
0:11:34 it about the difference between ideology and technical foundations.
0:11:37 I think that data ownership, one of the reasons we don't have it is because
0:11:41 it's hard to build apps that way.
0:11:43 It's not I think there are often companies that don't really have a
0:11:47 strong desire to build a SaaS data moat.
0:11:50 It just is kind of the default that is promoted by the way
0:11:53 we naturally build stuff.
0:11:55 And because it's so hard to build apps with data ownership,
0:11:57 we don't get that many of them.
0:11:59 And one of the I think most interesting potentials of local-first as an area that
0:12:04 I don't think has been fully realized yet is what if we could make it easier to
0:12:08 build apps this way rather than harder?
0:12:10 Right?
0:12:10 I think a lot of people in this space see that potential for this Radical
0:12:14 simplification of how we build stuff.
0:12:16 I think Peter, who was on the show see, you know, has sort of, Inspired
0:12:21 a lot of my thinking on that.
0:12:22 I think Martin Kleppmann, who was one of the coauthors of the local-first piece,
0:12:25 has given talks about how you sort of have this, uh, many layered cloud architecture.
0:12:30 And in theory, it could be so much simpler if you didn't have
0:12:33 to deal with all that stuff.
0:12:35 But, again, in practice, I think we're still getting there as a community,
0:12:39 and the reality is we're still maybe in the zone where it's harder today.
0:12:42 And so, You know, one of the ways that I thought about the Riffle project and,
0:12:46 you know, Nicholas and I and Daniel thought about it was what if the pitch
0:12:49 to an app developer for something like Riffle didn't start from ideology.
0:12:53 It didn't start from you care about giving your users their data and providing
0:12:58 interoperability, blah blah blah.
0:12:59 The reality is for many people, these are not the most pressing concerns.
0:13:03 What if we started instead from, hey.
0:13:05 Isn't it really complicated to build web apps?
0:13:08 What if it was just way simpler to build an awesome app that was fast, that was
0:13:12 really responsive, that stored all the data people care about, was easier for
0:13:16 you to develop faster, simpler, less ops maintenance stuff The data ownership
0:13:20 stuff just falls out as a consequence.
0:13:22 Right?
0:13:22 So a lot of our Riffle work was focused on, could we leverage the
0:13:28 power of databases and the power of the local-first architecture to cut through
0:13:34 a lot of the complexity of building modern web apps and make it simpler.
0:13:37 And in service of that mission, a few of our starting principles were
0:13:41 can you think of an application as one giant database Query.
0:13:45 What I mean by that is instead of writing a ton of normal code that produces
0:13:49 your UI, could you somehow push a lot of the work into a optimized database
0:13:54 that does a lot of heavy lifting and let it produce UI states for you.
0:14:00 Another thing that couples really well with that is fast synchronous reactivity.
0:14:04 So we have this idea that the data's already there on your device.
0:14:08 Throw away all the async crap that comes with web apps typically.
0:14:12 You know, React suspend network requests and work with the data
0:14:16 that's already there and simplify just talking to it and querying it.
0:14:20 And the last thing, our third principle was Can we take all these different
0:14:25 kinds of state that typically end up in different systems and throw them in one?
0:14:29 So your React UI state and your persistent local device local state
0:14:33 and your synchronized multi user state.
0:14:35 Can you just store them in one system?
0:14:37 And then You know, we like to think of it as maybe there's some setting
0:14:40 checkboxes on each bit of state that tell you should you share it or
0:14:43 whatever, but just have one system that's capable of subsuming all those roles.
0:14:48 And our hypothesis was that if you pulled all this off, it could feel really
0:14:51 awesome to build apps, and you would get Both better apps for end users because
0:14:57 they're fast and they work offline and they have all the local-first goodness.
0:15:00 But also you could get a better developer experience because you've removed a lot
0:15:04 of the layers of of traditional web apps.
0:15:06 So this was a lot.
0:15:08 And spoiler alert, that hypothesis that you've motivated, at least
0:15:12 in my very case paid off big time.
0:15:15 So the developer experience for me as a developer is better.
0:15:19 I hope that the end user experience will be better partially also
0:15:23 because I care a lot about performance and making things nice.
0:15:27 But to crack the chicken egg problem of, like, how do you even build better
0:15:33 apps with this in the first place?
0:15:35 What really was the the motivation and the the catalyst for me was
0:15:40 not all the ideals of local-first.
0:15:42 Sure.
0:15:42 I would love them, but that would not really make the difference for
0:15:46 me, like, start to work on an app and build in one architecture or another.
0:15:50 But the more Things I didn't have to do in the first place, the better.
0:15:54 The more I could really focus on working on the app.
0:15:58 And Just to simplify the entire stack your motivation to say, like, hey.
0:16:04 Actually, why do you even need a server?
0:16:06 Why do you need a back end for the thing?
0:16:08 Ultimately, the app is is working on your client.
0:16:11 How about that being the starting point?
0:16:13 That really spoke to me, and this drew me in as an app developer and
0:16:18 where I believe in in that mission.
0:16:21 So, yeah, I I couldn't agree more, and I would love to dig more into the various
0:16:28 principles that you've mentioned since I think there's quite a bit of, like,
0:16:31 interesting thinking behind each of those.
0:16:34 If I recap it correctly, I think it was simplifying the inner workings of
0:16:39 an app into Uh, basically, database queries, aka, like, complex SQL queries
0:16:44 or however else you wanna express it than the the synchronous uh, uh, graph
0:16:49 of typically how React works, etcetera, coupling that directly together.
0:16:55 And the third part oh, you gotta remind me about the third part again.
0:17:03 3 principles behind Riffle
0:17:04 Absolutely.
0:17:04 So, you know, the first principle I talked about is thinking of an
0:17:08 application as a database query.
0:17:10 Right?
0:17:10 And, actually, to throw a little more detail in there,
0:17:13 it's not one database query.
0:17:15 It's actually a graph of database queries with dependencies.
0:17:17 Now that starts to sound kind of fancy, but I think we all understand
0:17:21 this model from using spreadsheets.
0:17:23 The the metaphor, I think, to think about is can You make coding
0:17:27 a rich, complex application as simple as using a spreadsheet.
0:17:32 Spreadsheets dependency tracking of formulas and automatic
0:17:36 Dependency propagation is the bread and butter of spreadsheets.
0:17:38 It's one of the key features that makes them useful.
0:17:41 Right?
0:17:41 When you're using a spreadsheet and you update a cell, The contract
0:17:44 that Excel gives you is typically instantly or at least very quickly,
0:17:48 unless you're in a huge spreadsheet, everything will be up to date.
0:17:51 You're never worried about staleness.
0:17:53 You're never thinking about caching.
0:17:55 That complexity of dependency propagation is pushed down into the system.
0:17:59 Right?
0:18:00 And I would note, I think this is a lesson that we've definitely
0:18:03 learned in UI development.
0:18:05 You could definitely say that reactive UI frameworks like, React JS give
0:18:10 you something like this guarantee.
0:18:12 Right?
0:18:12 They you declare the UI you want and state and DOM are synced automatically.
0:18:18 The problem that I see is that often in our web stacks, this reactivity
0:18:22 is partial to one part of the stack.
0:18:25 So in in a common web application, your React UI, what it's
0:18:29 faithfully representing is not all the state in your system.
0:18:33 It's some view you currently have in this tab of some larger state that is,
0:18:38 you know, maybe primarily on a server.
0:18:41 And that introduces a lot of complexity because all of a sudden, you have
0:18:44 your nice declarative spreadsheety React thing, but then you have At
0:18:48 some point in your stack, you're gonna start hitting data fetching.
0:18:51 You're gonna start hitting sending rights back up to the server.
0:18:55 And, that's just an enormous source of mental model overhead, I think.
0:19:00 On the flip side there have been some interesting research projects which come
0:19:03 out from the other side, And people, for example, have literally tried to
0:19:06 make UIs in things like Google Sheets.
0:19:09 There's a really neat project by Ted Benson and collaborators called Quiltstep.
0:19:13 Literally, they use Google Sheets to power your entire back end and all your your,
0:19:18 you know, UI templating and essentially.
0:19:20 And the problem there is normal spreadsheets aren't really powerful
0:19:23 enough to do that, and they're not really architected the right way.
0:19:27 one way to think about what we're trying to do in Riffle is can you take
0:19:30 the beautifully simple model of just declarative dependency propagation, and
0:19:35 apply that to your entire data stack.
0:19:38 So your mental model as developer should be, I just have all the data locally.
0:19:42 I'm just writing a spreadsheet that turns it into a UI, and the
0:19:47 system will take care of the rest.
0:19:48 Full stack reactivity.
0:19:50 That's sort of principle number one.
0:19:52 That makes that makes total sense.
0:19:54 And I think you've Touched on also, like, the the third principle, which I remember
0:19:58 now is, like, the the unifying the UI states and sort of the the other app data.
0:20:03 And if you really think about it, where do you even draw the line between something
0:20:07 being UI state, something being app data?
0:20:10 There is sort of like a arbitrary line.
0:20:12 Maybe I think de facto in most today's web apps is the app data is, like,
0:20:17 what you fetch over something like React Query as your as your website
0:20:22 boots up or you have it pre hydrated.
0:20:25 And the app state It's more of like the in memory stuff that if you
0:20:29 reload the pages, poof, it's gone.
0:20:32 But, ideally, most of the things should really be treated in a very similar
0:20:36 way when I'm thinking of a native Mac app, for example, like Finder.
0:20:41 If I close a window and I come back, The same items are still selected.
0:20:46 I'm still, like, roughly in the the same position with the the folders that
0:20:51 I've selected and and the hierarchy.
0:20:53 And most web apps, I would actually suffer from that they they have, like
0:20:58 basically, they're suffering from dementia when you and insert something
0:21:02 in a form, and maybe you really need to look up every value and maybe the form
0:21:08 expires, you reload, everything is gone.
0:21:10 That's I think everyone has, like, those terrible user experiences.
0:21:14 And so treating everything as the same data, I think is is a kind
0:21:19 of a bold, but in my experience, like, the the right step.
0:21:23 And then if you can, Therefore, as a conclusion, kind of, like, have all
0:21:28 the data that you need to work with.
0:21:30 Have that locally.
0:21:31 It simplifies everything to such a degree that's really liberating.
0:21:36 So that that's sort of the the third the principle, and I think that's that's a
0:21:40 really bold one that is, I think the most provocative compared to the traditional
0:21:45 way of building web apps today.
0:21:48 Yeah.
0:21:48 Exactly.
0:21:48 You know, just like you said I think it can be a disrespectful user experience
0:21:52 to lose people's valuable input.
0:21:55 Even something like Scroll position or, like you said, intermediate
0:21:59 form state can be thing something that I put a lot of work into and
0:22:03 I don't wanna necessarily lose.
0:22:04 Right?
0:22:05 Now I think the deeper reason we end up with software like this
0:22:10 isn't that people don't care.
0:22:11 It's really that we've made it hard to do.
0:22:14 So, again, it's this technical defaults ruling, the user experience
0:22:18 situation that we talked about where, You know, imagine you're a product
0:22:22 manager and you get a request to please persist to this user input or
0:22:26 share this user input across users.
0:22:28 And the engineer says, oh, man.
0:22:30 That was previously React components data, and now I need to, like, rearchitect
0:22:34 where that data is stored and plummet through a bunch of places and Add a
0:22:38 database column for it, blah blah blah.
0:22:40 It's all this work.
0:22:42 And the goal of having a unified system for your state.
0:22:44 It's not that all states should be treated the same.
0:22:47 Definitely, some state, you know, should be user local, device local, and so on.
0:22:51 There, Yeah.
0:22:52 Some state you want to reset on new sessions by default.
0:22:56 So you can think of that as ephemeral.
0:22:58 But If you store it all in one system and manage it all in one system,
0:23:02 the goal is that it's becomes much easier to move between, these states.
0:23:06 Right?
0:23:07 one example I love is that someone who's working on a collaborative app
0:23:10 told me the state of whether a context menu is open in a spatial canvas app.
0:23:17 They actually realized they needed it to be shared because when I open a
0:23:21 context menu, my cursor is moving around.
0:23:22 If you can't see the context you I'm using, you don't
0:23:25 really know what I'm doing.
0:23:27 And so, I think that's a perfect example of blurring these boundaries
0:23:30 where something that Seems like clearly quote, unquote, local component state
0:23:35 in a traditional framing might actually need to be collaboratively synced
0:23:40 or, you know, maybe even persisted.
0:23:42 So that's what I see as the goal of having this unified state management system.
0:23:47 And in the Riffle work that's basically the the maximalist
0:23:50 view we took is no react state.
0:23:52 Right?
0:23:53 Every single bit of state in the system flows through this one database, and your
0:23:58 UI is just derived from that one place.
0:24:01 And You can always reason about essentially, the pixels on the screen
0:24:05 are a function of the database.
0:24:07 That's it.
0:24:07 Nothing else.
0:24:08 And we found a lot of neat qualities that that design led to along with
0:24:13 a bunch of interesting challenges.
0:24:15 And and I have to say, I was I was not on board with that when we got started.
0:24:20 I was so attached to like, everything is in React, and you have, like, things
0:24:26 inside of React components, etcetera.
0:24:28 And I remember Nicholas, he was, like, on the on the opposite side there.
0:24:33 And so, like, no.
0:24:34 No.
0:24:34 No.
0:24:34 Like, React should do as little as possible and, like, have your
0:24:37 state outside, and that didn't click quite back then for me.
0:24:42 But step by step, I came around to it, and I'm now also, like, on
0:24:46 a maximalist side of, like, most state should live outside of React.
0:24:50 And for example, for Overtone, there's a very simple case where
0:24:54 you for example, for the app for the playback of your player.
0:24:58 Like, you want your playback to happen to happen.
0:25:01 You wanna hear something.
0:25:02 Maybe you captured the the time code of the the current playback progress.
0:25:07 And even if the app is not open, You wanna still hear things.
0:25:12 If you reload the app you might just still want to recover at the
0:25:16 the correct playback position.
0:25:18 So all of that state is kinda headless.
0:25:20 So the state still lives on and, therefore wouldn't quite make sense to
0:25:25 have that be part of React, you say.
0:25:27 Sure.
0:25:27 You could say you have, like, a headless React component, but that's already
0:25:32 feels a bit backwards at that point.
0:25:34 So Once I've started embracing that, it was hugely liberating, and now I can
0:25:40 start to reason about my state outside of the context of React components in
0:25:46 that outside of reactive view state.
0:25:49 So That took me a bit to get around to it, but I'm fully on board
0:25:53 with that principle now as well.
0:25:55 you know, I'll I'll mention, I don't think we're the we're not
0:25:57 the only ones who've of this right.
0:25:58 I think um, the idea of having a reactive state graph separate from
0:26:02 your UI tree is something that other web dev libraries have explored.
0:26:07 MobX, Recoil, arguably, even something like Redux, I think shows some of the
0:26:11 power of having this sort of centralized, outside of your UI tree state management.
0:26:16 So, there's definitely, I think some broad recognition that this
0:26:21 can be a powerful pattern for for some of the reasons you mentioned.
0:26:25 You know, but I'll also mention, like, there are some challenges.
0:26:27 Right?
0:26:27 And, I mean, You've encountered some of these.
0:26:29 I'd be curious to hear your take.
0:26:30 Some of them that I think immediately come to mind for people are, one,
0:26:35 Performance is always an easy one.
0:26:37 Like, typically, we expect button hovers and, you know, selection
0:26:42 changes to happen at 60 120 Hertz.
0:26:45 And we don't expect things like loading thousands of rows of
0:26:50 data from our core data store to necessarily happen at the same speed.
0:26:54 And so that mismatch can be an an an interesting thing to Manage.
0:27:00 Our motto Nicholas's motto to some extent is just make everything fast.
0:27:03 If everything goes at 120 FPS, then don't have a problem.
0:27:06 Right?
0:27:07 Easier said than done, but I think a fun goal to shoot for.
0:27:11 So far, most mostly what we've been talking about is
0:27:13 conceptually a simplification.
0:27:16 Getting on board with that different way of thinking about
0:27:19 your your app on a conceptual basis.
0:27:21 And I think for me, it took a little bit of time to ease into that,
0:27:26 and then, be confronted with the new challenges that are raising.
0:27:29 We we'll we'll go into the challenges in a second since
0:27:32 We've encountered quite a few.
0:27:34 But I think we didn't quite touch too much on the the second principle you've
0:27:38 motivated before, which is, uh, the synchronous reactivity, and, like,
0:27:43 that should be as fast as possible.
0:27:45 We'll we'll go into the performance challenges there in a bit.
0:27:48 But when you've said synchronous reactivity as opposed to
0:27:54 asynchronous reactivity, what what are the why is this a principle.
0:27:58 Like, what is so important?
0:27:59 Can can you motivate the the problem and the status quo and
0:28:02 how Riffle is challenging that?
0:28:04 . Yeah.
0:28:05 Absolutely.
0:28:05 This is a really important principle to me.
0:28:08 I think when we Build web apps that have a server.
0:28:11 We assume a lot of asynchrony at many points.
0:28:15 So, obviously, going on the network to a server is a point of asynchrony.
0:28:19 Even often loading data from you know, like a local disk.
0:28:23 Right?
0:28:23 We think of as an asynchronous operation.
0:28:25 And because of that, UI development typically has a lot of reasoning
0:28:30 about asynchrony, which is a really hard thing to do as a programmer.
0:28:34 For example, you might have a UI where There are loading states.
0:28:38 And everywhere throughout your app, you need to think about, am I still
0:28:42 loading, or do I have the data?
0:28:43 And maybe you have a UI that's composed of different sections, and some of them
0:28:47 might be loaded already and others aren't.
0:28:49 You might have intermediate states to reason about.
0:28:51 Like, After I select something in a sidebar, there's some time
0:28:55 where the main other pane of the app hasn't reacted to that yet.
0:28:59 So It might still be showing the old thing that was there before I selected
0:29:02 the new thing, or it might be showing a loading state while I'm loading the data.
0:29:06 And my experience at least personally has been that We kind of accept
0:29:10 this as just the way it is in UI development, but it's terrible.
0:29:13 It's it's a lot of overhead to reason about.
0:29:16 There are many, many more states your system can be in when you have asynchrony.
0:29:21 Right?
0:29:21 And the the vision for synchronous reactivity is let's make it again
0:29:25 feel more like a spreadsheet.
0:29:26 So the way I want my UI to feel is that when I select something, on the
0:29:31 next tick, on the next frame, every pixel in the UI has fully updated
0:29:35 to reflect that new user input.
0:29:38 And that's obviously good for users because they didn't have to wait
0:29:42 But it's also good for developers because you don't have to reason
0:29:43 about the intermediate states.
0:29:46 You can sort of think transactionally where The state of the world changes,
0:29:50 and, transactionally, we update a bunch of derived views and queries of that state
0:29:56 which end up in the pixels of the UI.
0:29:58 And there was never any intermediate state exposed.
0:30:01 And it's It's sort of a subtle point to get across, but my experience has been
0:30:06 that when you work in a system like this it just Cuts a lot of complexity.
0:30:11 Yeah.
0:30:11 It's not just a simplification in terms of architecture that we collapse
0:30:16 the stack from back end and API, etcetera, and all into the client.
0:30:21 But that's, like, one giant dimension of complexity.
0:30:26 But there is a second, much less talked about vector of complexity,
0:30:31 which is like, how is how are things related to each other?
0:30:34 Can they be synchronously related or can or do they have
0:30:38 to be asynchronously related?
0:30:41 And this is where I think we're getting into the area of distributed systems.
0:30:45 I'm sure we'll talk a lot more in future episodes about broader distributed
0:30:50 systems where they're really needed, where you have You're you have a device.
0:30:54 You're in the US.
0:30:55 I'm in Europe, and so we are distributed.
0:30:57 However, if we are starting to treat a local React app that's
0:31:01 running in single thread And we're coordinating the various React use
0:31:06 states things with React useEffect.
0:31:08 We've kinda accidentally created a distributed system that's It's really hard
0:31:13 to tame and where it is, there can be many unintended bugs and just starts becoming
0:31:19 a lot more complex to to think about.
0:31:22 So this is the other part where I think it really it con where Riffle,
0:31:27 again, liberates so much and simplifies and frees us of the the accidental
0:31:33 distributed systems problem that we have in most modern web apps.
0:31:37 I, you know, I, think to 2 principles that I think about one is don't
0:31:43 make something a distributed system if it really doesn't have to be.
0:31:47 And 2 is if it has to be, someone smarter than me should
0:31:51 do the distributed systems part.
0:31:53 Right.
0:31:53 Really, we should be providing abstractions that allow us to not think
0:31:58 about the hard parts most of the time.
0:32:01 Even, you know, I'm not a CRDT expert, but I know quite a bit about CRDTs and
0:32:05 have done some work on developing CRDTs.
0:32:07 But and so I, you know, I Probably know more about CRTs than a lot of
0:32:11 app devs, but I don't wanna think about distributed systems when
0:32:15 I'm building some product feature.
0:32:17 Right?
0:32:17 And I think That is clearly one of the core challenges in the local-first space
0:32:21 that I think a number of projects and companies are trying to address, which
0:32:25 is what are the Good abstractions you can expose to app developers that allow
0:32:29 them to reason about tricky distributed systems problems in a in a straightforward
0:32:36 way without needing to turn on your sort of distributed systems brain as much.
0:32:42 Yeah.
0:32:42 Exactly.
0:32:42 And I I think the Being confronted with distributed systems problems there is
0:32:48 much more common than most app devs think.
0:32:52 So most of the time when in React, you use useEffect that that hook to, yeah, somehow
0:32:59 tame your local state, Then you already have a distributed systems problem.
0:33:04 Things like React suspense and and so on.
0:33:07 That that's kind of like a a solution to address partially the the distributed
0:33:12 systems nature that you have in React.
0:33:14 So, I remember having when React suspense, and all of those primitives came out.
0:33:19 I was kinda wondering, how does that fit into our Riffle world here?
0:33:24 would Riffle at some point support react suspense, and then at
0:33:28 some point, it clicked for me.
0:33:30 No.
0:33:30 That's, like, addressing a problem that we've already ruled out
0:33:34 before, and we don't even have to think about that complexity at all.
0:33:39 And yeah, this is, like, entirely different approach to addressing
0:33:44 the problems that React, Suspense, etcetera, is addressing in a, I would
0:33:49 say, in a much simpler overall picture.
0:33:52 Yeah.
0:33:52 I think maybe to avoid overclaiming here, I'll say there are some trade offs.
0:33:57 Right?
0:33:57 I think Often, these approaches that involve concurrency are they're optimizing
0:34:02 for either the reality of something like a slow network request or for
0:34:05 sort of worst case performance making sure that you keep some things running
0:34:09 fast even if other things are slow.
0:34:11 And I think there's a different approach you can think about, which is, In a
0:34:15 nutshell, just make everything fast.
0:34:17 And as long as everything's fast, it'll be simple.
0:34:19 If things get slow, you might end up with a worse experience than you
0:34:22 might have in a system that, you know that is designed around concurrency.
0:34:26 For example, maybe in a in a Riffle style approach, if you're trying to
0:34:31 synchronously paint frames and something is slow, you might just end up with a ton
0:34:35 of dropped frames and sort of a frozen UI.
0:34:38 Whereas in a system designed to account for that with better concurrency support,
0:34:42 you might have, you know, a loading spinner and some things remain responsive.
0:34:46 And so I think Part of the challenge is figuring out how to
0:34:49 how to balance those 2 approaches.
0:34:52 And as we've seen in our work, I think sometimes you do need to account for
0:34:56 the reality of sometimes you have to make a network request because, you
0:34:59 know, you're talking to an external service or something, and you do
0:35:02 need ways to model concurrency.
0:35:03 It's just that I think We don't need to, I guess, throw the
0:35:07 baby out with the bathwater.
0:35:08 Like, not everything needs to be concurrent, and some things can be fast.
0:35:12 I think I don't have much game dev experience, but I my understanding is
0:35:15 that a lot of video games, essentially, you know, they paint frames, and
0:35:19 they make sure every frame is fast.
0:35:20 And they don't drop frames because they make sure the frames are fast.
0:35:23 Right?
0:35:28 What is Riffle?
0:35:28 So we've been referring to this thing, Riffle, now throughout this conversation.
0:35:33 And clearly, you and I are well aware of what Riffle is, And you've motivated
0:35:38 what Riffle is supposed to be through the principles and sort of what you were
0:35:43 hoping to achieve with it in theory.
0:35:45 But in reality, what is Riffle?
0:35:49 Riffle was a research project.
0:35:52 And so our primary goal was to try to get some of these ideas into people's
0:35:55 heads, but we did need a way to test them out and see, is this even possible?
0:35:58 Is this really a good idea?
0:36:00 And to do that, we built a simple prototype reusing existing
0:36:04 parts as much as possible.
0:36:06 Namely, we we built on React JS as a view templating layer, and we built on
0:36:10 SQLite, which is a popular relational database that can be embedded on
0:36:15 devices and in the browser through Wasm.
0:36:18 And what Riffle is, technically, is a layer that sits between React and SQLite.
0:36:24 And what it does is it lets you specify a bunch of, SQL queries that turn a
0:36:29 SQLite database into a React UI, and it gives you some hooks in your React
0:36:33 UI to do things like query data in a component or make writes to your database.
0:36:38 And you can think of it sort of as a state management framework built on
0:36:42 top of SQLite for React that tries to realize some of these ideals we
0:36:46 just talked about in a concrete way.
0:36:48 Right?
0:36:49 Now, again, this uh, was just sort of a prototype to explore the ideas.
0:36:53 And one challenge we faced early on was how do we really test out if this works
0:36:59 in the kinds of context we care about, which are complex data intensive apps with
0:37:04 pretty large amounts of data relatively speaking and interesting schemas
0:37:08 and strict performance requirements.
0:37:10 To do apps are kinda boring and don't really address any of those
0:37:13 and that's actually how we ended up collaborating with you on this.
0:37:16 So, Maybe do you wanna tell the audience a bit about how you got involved?
0:37:21 Yeah.
0:37:21 Totally.
0:37:22 We've been in touch since You worked on Cambria, and I think we've
0:37:26 been catching up once in a while.
0:37:28 And then at some point I learned about you thinking about this this broader
0:37:32 space and rethinking state management.
0:37:36 With my background of Prisma, I'm obviously no stranger to
0:37:40 thinking about state management.
0:37:41 I've been thinking about how to make it simpler, typically
0:37:45 more in a back end context.
0:37:47 And for me, this was really the the interesting foundation
0:37:51 that you said, like, hey.
0:37:52 Why why do you even want the back end?
0:37:54 You wanna build an app.
0:37:55 You didn't quite say it like this, but this is the way how I how I had sort of
0:37:59 picked it up and framed it in in my head.
0:38:02 And that was a really stimulating idea for me to say, like, actually, yeah,
0:38:06 let's bring the database into the client.
0:38:09 I was using a lot of MobX and Redux and those sort of things in
0:38:13 the past, and they they worked.
0:38:15 They made me productive, but they didn't have the properties that you mentioned
0:38:19 before that state wasn't persistent.
0:38:22 It fundamentally treated state and data differently.
0:38:26 I would fetch my data from from a back end . It was a very Interesting idea for
0:38:32 me to have the data all there locally.
0:38:35 So and I was thinking about building my own music app for for quite a while, and
0:38:41 Riffle seemed like the perfect catalyst for me to really take a stab at it.
0:38:46 And the only way how I could could really make progress on such a
0:38:51 big effort, mostly by myself, was to cut scope as much as possible.
0:38:58 And that's that was a very simple math to say, like, hey.
0:39:02 We can cut the scope from the back end categorically, and I can
0:39:07 focus all of my energy and time on building the best client possible.
0:39:10 And a music client is no nobody talks about, oh, that's the best music back end.
0:39:16 People want the best music app.
0:39:18 And that has led me to become the first design customer, does it first design
0:39:24 partner for Riffle, and that made a a brilliant combination, I think.
0:39:28 Yeah.
0:39:28 I think one alignment and values that has really helped here is that, you're trying
0:39:33 to build a music app for power users.
0:39:35 Right?
0:39:35 And I think power users often care about things like data density, latency.
0:39:40 Um, you know, it's funny.
0:39:42 When you look at apps like Spotify, for example, and we've we've talked about
0:39:45 this many times over the course of this collaboration, Spotify, their desktop
0:39:50 app is quite slow often and also quite network reliant to a surprising degree.
0:39:56 Many page transitions that we've encountered even for data that is already
0:40:01 locally cached, for some reason has a lot of network access going on, and you
0:40:05 often get multi second loading screens.
0:40:07 And I think for serious Music people like DJs you know, some of
0:40:12 your target audience for Overtone.
0:40:14 I think of it a bit the same way that, serious email people like apps like
0:40:18 Superhuman that respect their time and make them really fast at email.
0:40:21 A serious music fan wants an app that feels good and not some, you know, loading
0:40:27 screen filled consumer feeling app.
0:40:29 Right?
0:40:30 And I think that was really aligned with some of the goals that we wanted
0:40:32 to hit with the Riffle approach is, Basically, can you just throw all the
0:40:36 music metadata in a SQLite database use relational queries, which are really
0:40:40 good fit for things like joining together albums and artists and tracks, things
0:40:44 like that, and power, the kind of power user experience that you wanted to build.
0:40:50 Exactly.
0:40:50 And I think this was literally also, like, the starting point for our
0:40:54 first prototypes to do a data model.
0:40:57 I'm kinda employing similar ideas that you use in back end development where
0:41:01 you first lay out your different tables.
0:41:03 In this case, I didn't actually create a user table.
0:41:06 I still don't have a user table since the app just runs locally, but
0:41:10 I did create a tracks table and a albums table and a playlist table.
0:41:14 And we threw that together.
0:41:16 And within the shortest period of time, we had a fully
0:41:20 functional, very crude music app.
0:41:23 And step by step yeah, we we I think we've been looking for reasons why it doesn't
0:41:29 work, and we never found that reason.
0:41:31 And then the months passed, and the collaboration now lasted, I think,
0:41:36 for for almost, like, 2 years.
0:41:38 And you've confirmed the hypothesis that you had with your with your
0:41:42 PhD thesis at MIT for for Riffle.
0:41:45 this made for a brilliant partnership.
0:41:47 Yeah.
0:41:47 I think you know, one of our goals with working with you
0:41:51 was that first of all, yes.
0:41:52 Like you said If it went well, which I think it has gone better
0:41:55 than we expected, then it's more valuable confirmation, right, that,
0:41:59 like, a real serious app works with this than, like, a to do MVC demo.
0:42:04 So I think that was one thing.
0:42:05 But, also, the the primary goal was not that.
0:42:07 The primary goal was uncover what the real challenges are of building
0:42:11 real software in this model.
0:42:12 And maybe you could talk a bit about you know, throughout the project,
0:42:16 you've been a a very useful critic and sort of have pointed out a lot
0:42:22 of the the problems and challenges that come from working in this way.
0:42:26 And I'm curious what your take has been on what the biggest problems
0:42:32 Riffle Challenges
0:42:33 Totally.
0:42:34 So I as a person, I like pioneering things.
0:42:39 So I have probably a higher risk appetite, and I can deal with, like, a
0:42:44 couple of more paper cuts temporarily than someone else who is just looking
0:42:50 for the the best well trodden path.
0:42:53 So, this this made me a viable partner here as opposed to to someone who
0:43:00 only touches the technology that is already well proven over the years.
0:43:05 And so I was rather looking for an advances on a in a broader scale,
0:43:11 like, on a architectural level and just something that in the long run can help
0:43:16 me reduce, and avoid complexity as much as possible even though temporarily
0:43:23 possibly dealing with enduring a few more paper cuts here and there.
0:43:28 And given that I am both comfortable in the shoes of an app builder as well
0:43:33 as in the shoes of a tools builder.
0:43:36 I think I was uniquely positioned to switch between those those 2 modes
0:43:41 to unblock me etcetera, and to have enough imagination to understand,
0:43:47 oh, this is not a categorical flaw in the way how things are right now.
0:43:51 We can fix that through better tooling or refactoring of the API.
0:43:56 So this I think it was kinda in in waves where there was some waves of
0:44:02 immense productivity, and things have been working much better than expected.
0:44:07 And at times, we've hit some walls of doubting, is this ever gonna work?
0:44:12 Are those principles, the the ones that we've picked Are those the right ones?
0:44:18 Is it a good idea to keep our entire state graph synchronous,
0:44:23 can we make it fast enough?
0:44:24 So we've been hitting those challenges, And the ones that are most memorable
0:44:29 to me was really around performance I can motivate it from my perspective.
0:44:35 I've set it for myself as a challenge.
0:44:38 I want to build a music app That's fundamentally built with
0:44:42 web technology that should still feel as fast as a native app.
0:44:47 So that means If you're now looking at a 60 hertz display or a 120
0:44:52 hertz display, if you scroll, if you interact with the app in some way,
0:44:57 It should work without frame drops.
0:45:00 And given that all state management, everything everything that changes
0:45:05 on your screen, is fundamentally a result an implication of state change.
0:45:11 And that this happens smoothly, it needs to happen in a 120 hertz.
0:45:17 So, that basically gives you per frame, yep, less than 4
0:45:21 milliseconds to do everything, state change, render, react, etcetera.
0:45:26 So performance was always top of mind, and I think that where we've been going
0:45:31 back and forth is it ever gonna work?
0:45:33 Oh, yes.
0:45:33 We can make it work.
0:45:35 And that has kept us busy for for quite a while, and I think we can go more
0:45:39 into depth what that's what that meant.
0:45:42 But that's that's the top challenge that comes to mind besides a few, like, paper
0:45:47 cuts and also figuring out, like, how do we the the starting point, even though
0:45:52 this is a local-first podcast, Riffle Up to this point, it was mostly local only.
0:45:58 And just in a later point, we're making we're introducing the the collaboration
0:46:04 aspect to make it truly local-first.
0:46:07 But yeah, I think performance been the the major challenge that we've been facing.
0:46:11 You know, the way I think about performance in this stack is that the role
0:46:15 of the database is to make things fast.
0:46:18 What a database does is you give it nice declarative queries, And it figures
0:46:22 out how to make them fast because some smart people worked very hard,
0:46:25 and you didn't have to do that work.
0:46:26 Right?
0:46:27 And I think a lot of the challenges we've hit basically boil down to, um,
0:46:31 we don't quite have the right database for this use case or a shape yet.
0:46:36 A lot of UI stuff, I think, boils down to the problem of incremental computation.
0:46:42 Incremental computation is a is a pretty broad, you know, computer science term.
0:46:45 That basically means I have a function and the input changes a little bit.
0:46:50 Can we figure out how the output changed without starting over
0:46:54 from scratch on the fresh input.
0:46:56 Right?
0:46:57 And this is a problem that people have approached from a number of sides.
0:47:00 The sort of programming languages community has approached this problem.
0:47:04 You know, incremental computation is the keyword that can highlight
0:47:07 a lot of the work there.
0:47:08 But the databases community has also thought a lot about this problem.
0:47:11 They Often call it incremental view maintenance.
0:47:14 You know, an example might be I have a big join on thousands of
0:47:18 rows and I add one row to the table.
0:47:20 How does the result change.
0:47:22 Right?
0:47:23 And I think a lot of the reasons we've had performance challenges is that our
0:47:27 technical stack has mostly been based on SQLlite which is a database that is sort
0:47:32 of built for UI, but is not incremental.
0:47:34 We did some explorations with a database called SKDB which is actually a SQL
0:47:40 database designed to be incremental, And that yielded some pretty
0:47:44 interesting and promising results.
0:47:45 We actually published a paper at the UIST conference, which talked
0:47:49 about some of our positive results from working with that technology.
0:47:52 I think, ultimately, you ended up deciding to stick with SQLite for the
0:47:56 production Overtone app because you wanted a sort of a more, you know, mature
0:48:00 database to work with as your foundation.
0:48:02 But the way I see the problem is really that we sort of need the right database
0:48:07 that has the right primitives to make UI development of this sort fast.
0:48:11 And maybe the perfect database for that doesn't quite exist yet.
0:48:15 But I think that's that that's the missing piece and the ultimate, you
0:48:19 know, decade ahead if this really panned out, version of the stack, I
0:48:24 think, would include that database.
0:48:26 Yeah.
0:48:27 Totally.
0:48:27 And I I think it's a matter of, like, now it's early 20 24.
0:48:31 Who knows?
0:48:32 Maybe next year, the the world has already moved closer into the
0:48:36 direction that is a good foundation for for something like Riffle.
0:48:40 The the audience shouldn't have the takeaways like SQLite is slow.
0:48:44 Quite the opposite.
0:48:45 I think this was the insight that SQLite is incredibly fast and remarkably
0:48:51 capable was the foundation to embark on this project in the first place.
0:48:55 Nicholas has been through his prior work at Apple, Done a lot
0:48:59 of work related to SQLite and I think related to to FoundationDB.
0:49:04 And that was one of the key insights that we say, like, Actually, SQLite is
0:49:08 so fast and now can be embedded into web applications through Wasm that it
0:49:13 can be a replacement technology for some for, like, MobX and and Redux.
0:49:18 So before WASM, etcetera, and all of the optimizations that have gone into that.
0:49:23 This wouldn't have been possible.
0:49:25 But now that's possible this allowed for a better foundation for to deal
0:49:31 with complexity since if you build something with MOBX, etcetera, you
0:49:36 still need to wrangle all of your JavaScript objects by yourself.
0:49:40 Like, if you want to have, like, a filter or map or group by, you gotta
0:49:44 implement all of that yourself.
0:49:45 And the app user experience that I want to enable with Overtone
0:49:52 is this will be very common.
0:49:54 Think about it, like, in Notion when you have the tables feature and
0:49:58 you have You configure a few sorts.
0:50:01 You configure some group buys.
0:50:03 You have different views.
0:50:04 Those are all things that get very boilerplatey to implement
0:50:08 with JavaScript, and those are all things databases are super good at.
0:50:13 And this is what got me over the fence to say, like, actually, let's
0:50:18 embrace what the database is good at, and build a bit of, like, the
0:50:22 React nice to have things around it.
0:50:25 And I think the the core of what Riffle is is really like that combination
0:50:32 of a underlying SQLite database and put a reactivity system on top of it.
0:50:37 And reactivity system in that regard is, like, how React Components compose,
0:50:42 and if one thing changes, then the other thing updates to the minimal degree.
0:50:47 That's what Riffle is doing.
0:50:48 And A very simple idea with many implications and remarkably powerful.
0:50:54 Yeah.
0:50:55 You know, I I think you're you're right, and it extends beyond performance too.
0:50:59 I think when you think about the requirements of what a UI needs, I
0:51:03 think that when you remove all the layers that typically sit between a
0:51:08 back end relational database and a UI.
0:51:10 Those layers were there for a reason.
0:51:13 They were solving some problem, and you end up needing to find
0:51:15 ways to solve those problems.
0:51:16 So, often, UIs need tree shaped data to hydrate a tree of a UI.
0:51:23 Right?
0:51:23 Often front end developers don't already know SQL.
0:51:28 And often, actually, SQL, we found can be a mediocre language for
0:51:32 certain parts of UI development.
0:51:33 It often just feels a little bit too low level or something.
0:51:37 Obviously, you know, having worked on Prisma, you, I think, have experience with
0:51:40 some of those problems in the back-end context, but I think some of them become
0:51:44 even more apparent in a front-end context.
0:51:47 Some of the you know, I I don't wanna necessarily necessarily say the ORM
0:51:52 word because I think that's a touchy topic, but I think some of the roles of
0:51:56 associating different kinds of models with each other and things like that,
0:51:59 there's sort of, sometimes feels like there's a need for for that kind of layer.
0:52:03 Right?
0:52:04 I think you know, in our technical prototype stack, what we ended up doing
0:52:08 was stuffing a GraphQL layer in there between SQLite and the UI you know,
0:52:13 because you're a GraphQL expert and that helped alleviate some of these problems.
0:52:17 But, again, I think I see and I think we we see that more as a as a shim.
0:52:22 Right?
0:52:22 Like, sort of, a layer that shouldn't need to be there, but
0:52:25 it's there because the database didn't quite have the right shape.
0:52:28 And I think some of that reflects that maybe no one has ever designed a database
0:52:34 specifically for this role of being embedded so closely to a user interface.
0:52:39 Yeah.
0:52:40 Totally.
0:52:40 I mean, I I think about it this way.
0:52:43 We wanna take one or 2 steps forward, But in order to do that, we also
0:52:48 had to take a step backwards.
0:52:50 And the the step forward that we took is that we get the power of a database,
0:52:54 that we get all of those semantics that a database give us, whether it's group by
0:52:59 and sorting and nested selects, etcetera.
0:53:03 All all of those great things that would really um, freeze up our work
0:53:07 and make some things that we would need to imperatively implement in
0:53:12 JavaScript makes that declarative.
0:53:14 However, the steps that we're taking back is that, In SQLite, it's quite limited.
0:53:20 It doesn't have many data types.
0:53:22 It doesn't even have a Boolean type, etcetera.
0:53:24 So you need to do a lot of translation between the way how SQLite understands
0:53:29 the world and how someone who's, like, a a more sophisticated
0:53:35 TypeScript developer as me.
0:53:37 I'm used to having all of my fancy TypeScript types.
0:53:40 And to create a good translation layer between that embedded SQLite database
0:53:45 and what I want in my JavaScript now we gotta reinvest in that.
0:53:51 And so you've mentioned GraphQL.
0:53:53 That was sort of a technology that I was quite familiar with from prior
0:53:58 work, and I think that's sort of a temporary usage until we we teach SQLite
0:54:04 some of the tricks that TypeScript already knows and until we create a
0:54:10 a better mapping in in between that.
0:54:13 But that's certainly been another, For me, less of a challenge, but it still requires
0:54:18 quite a bit of work to make SQLite feel and a native in a TypeScript setting.
0:54:24 Some people, I think, believe that one of the sort of original sins
0:54:28 of computing, so to speak, is that programming languages and databases
0:54:32 really split into separate fields.
0:54:34 And the idea of, like, the system that you use to wrangle your runtime
0:54:38 in memory state And the system that you use to persist and query your
0:54:41 persisted state are so separate in the way you think about them.
0:54:45 You historically, you have systems like small talk, for which
0:54:49 have much less of a separation.
0:54:50 They're closer to you have objects in your language and you're just saving
0:54:55 those objects and reloading them, And there's much less of a gap there.
0:54:59 Right?
0:54:59 And something I found interesting about some of the more recent work you've
0:55:02 been doing in this space is you know, you've been working on tools that make
0:55:06 it easier to close that gap and let you think about the kinds of in memory
0:55:11 objects that you wanna be thinking about in your program, and turn those into the
0:55:16 things that you wanna be storing in your database and querying and vice versa.
0:55:20 And I think that's again one of the really interesting challenges in
0:55:23 this space is handling that impedance mismatch between a relational database
0:55:29 and, your TypeScript code or whatever.
0:55:31 Do you wanna talk a bit about that area you've been working on?
0:55:35 Yeah.
0:55:35 Totally.
0:55:36 I think this is also, like, a for for you, like, a broader umbrella
0:55:40 of of your work that contextualizes everything you've been doing over the
0:55:44 past few years and will probably still be the foundation for for the the
0:55:48 years to come as Malleable software.
0:55:51 For me a common theme seems to be all around schema management and
0:55:56 kind of fighting that impedance mismatch from different contexts.
0:56:02 And so I've been trying uh, to make that simpler in the context of of Riffle.
0:56:07 And in a way that nicely brings us back to the initial topic that we talked about
0:56:12 with Cambria since it's not just Already hard to deal with that appease mismatch
0:56:18 between your application, types that are expressed, for example, in TypeScript
0:56:23 and the way how a database thinks about it how it stores the the data.
0:56:28 But both can also change over time.
0:56:31 You wanna implement new features, etcetera.
0:56:33 So your database schema might change, Your app types change, so you also get a
0:56:39 deal with with schema change management here, which hopefully, we'll we'll
0:56:44 get to bring some of the goodness of Cambria in into the fold here as well.
0:56:49 Schema migrations is still Sort of a untamed problem, but we're we're
0:56:55 getting ahead of ourselves here.
0:56:56 So, if Someone in the audience thinks about, okay.
0:57:01 Riffle sounds great.
0:57:02 Can I use it?
0:57:04 You've been mentioning that you've been doing this as part of your PhD at MIT.
0:57:11 Where's Riffle now?
0:57:12 Yeah.
0:57:12 I get asked this question from time to time, so it's probably
0:57:15 good to provide an update.
0:57:17 So Riffle was a research project fundamentally.
0:57:19 The goal was to explore this idea and see if it was possible or a good idea.
0:57:24 It ended up being part of my PhD thesis, and I wrapped up grad school.
0:57:29 And at this point, sort of the the core Riffle project itself is is,
0:57:34 I would say, it's basically over.
0:57:36 We published a paper at the UIST HCI conference this year that
0:57:40 summarizes some of what we learned.
0:57:42 However the ideas are not, dead.
0:57:44 So, actually, you, Johannes, have been sort of carrying the torch forward
0:57:48 to some of these ideas and building them into a new library that you're
0:57:51 building, that has sort of replaced Riffle as the foundation for your work
0:57:56 on Overtone and so maybe you could talk a bit about your ongoing work there.
0:58:01 Yeah.
0:58:01 Totally.
0:58:02 So for me, what's been the most valuable thing over the course of
0:58:07 the past few years in terms of our collaboration is, a, that collaboration,
0:58:12 that partnership, and really doing a lot of research and exploratory work
0:58:17 together and then, yeah, b, having sort of, like, a prototype implementation
0:58:23 of what that could look like.
0:58:24 But it was really a prototype implementation that could prove
0:58:29 out those different ideas.
0:58:31 But to make actual progress with Overtone, I was running into a lot of
0:58:37 the limitations of the those prototypes where it was working on the happy path.
0:58:42 The not so happy path was unexplored and started causing
0:58:47 a bunch of problems for me.
0:58:49 So this caused me not to throw out the baby with the bathwater and reach back
0:58:53 for something like MobX or or Redux.
0:58:57 But it has led me to yeah, where put on my dev tool builder hat again.
0:59:04 And I started productionizing the ideas of Rifflemore, and I did
0:59:11 that under a new umbrella, which is a library called LiveStore.
0:59:15 It is not yet open source.
0:59:17 I'm hoping to get it open source sometime this year.
0:59:22 Right now.
0:59:23 If someone is curious to give it a try it is possible right now if
0:59:28 you are sponsoring the the project.
0:59:30 That's a topic for another day.
0:59:32 I'm hoping to make development of LiveStore sustainable.
0:59:36 And through GitHub sponsorship is one way how I'm currently planning to do so.
0:59:41 But, yeah, the ideas of Riffle live on and possibly multiple projects in the
0:59:46 future, LiveStore being one of them.
0:59:49 I'm sure your work on Riffle has inspired more folks to implement
0:59:53 similar ideas in the future.
0:59:55 So that's that's what I can contribute right now.
0:59:59 Yeah.
0:59:59 I'm really excited to see where your work with Livestore goes.
1:00:02 You know, I think You have some some cool stuff cooking and places you're trying
1:00:06 live store, and I'm sure you're gonna keep learning a ton more about where
1:00:11 this approach works and new challenges.
1:00:13 Right?
1:00:13 But it's very exciting to me that these ideas are living on in your work.
1:00:17 Yeah.
1:00:18 And so Geoffrey and I are still touching base every so often, and I share my
1:00:26 updates and progress on LiveStore.
1:00:28 And Now step by step I get to make progress on some parts that we haven't
1:00:34 quite got to yet while working on the Riffle research project and
1:00:40 where I get to unlock some progress where Geoffrey thinks, oh, yes.
1:00:46 Finally, we're getting to that point that we haven't gotten
1:00:50 to while working on Riffle.
1:00:52 And I now I'm I have to face all of those bigger, implications that
1:00:59 we've been well aware of, but we just haven't gotten to them yet, whether
1:01:02 it's schema migrations or making the technology actually local-first.
1:01:07 So introducing syncing and collaboration capabilities, and I wanna do so
1:01:12 in a way that is compatible with different kinds of syncing approaches,
1:01:16 whether it might be something like electric SQL or possibly something
1:01:20 like Auto Merge or Yjs in the future.
1:01:23 So there in a way, we've come really far, and in a way, we are just getting started.
1:01:28 So I'm I'm super excited about that.
1:01:31 So we've been talking a lot about SQLite, Riffle, LiveStore, But you've
1:01:37 since moved on to to new projects.
1:01:40 So and I think you're still at Ink and Switch even more so than for I
1:01:44 think you're now full time at Ink and Switch and are already in new projects.
1:01:53 Geoffrey's new projects
1:01:53 Yeah.
1:01:54 So since finishing my PhD earlier I guess last year over the summer, I've now
1:02:00 for a few months been full time at the Ink and Switch Research Lab where Peter
1:02:03 Van Hardenberg again is the director.
1:02:06 You know, they're where a lot of the look they they coined the term local-first.
1:02:10 Right?
1:02:10 And so they're what got me into this space in in some sense, and I'm
1:02:14 Really excited to be full time there.
1:02:16 I'm leading our malleable software research track now.
1:02:19 And so that's basically a research track where we explore these ideas
1:02:22 around customizable software.
1:02:24 And we really are interested particularly in the intersection
1:02:28 of the local-first architecture and the malleable software agenda.
1:02:32 And we have a lot of ideas for how those 2 things can be kind of complementary
1:02:36 to each other as I described earlier.
1:02:39 We have some fun experiments brewing.
1:02:41 one thing is that we're increasingly trying to use our own local-first tools.
1:02:46 So one Little project I worked on in my past few months there is
1:02:50 spinning up a writing tool that we use internally at the lab.
1:02:54 We used it to write our last essay that we published and that's based on the auto
1:02:59 merge stack that the lab has developed.
1:03:01 And it's a really nice way to sort of, Again, just get some authentic
1:03:05 experience using the stack and understanding the challenges and
1:03:08 the and the and the good stuff.
1:03:10 Right?
1:03:11 We're then kind of using that stack as a foundation to explore a lot
1:03:16 more speculative stuff going forward.
1:03:18 You Yeah.
1:03:18 Some of the ideas we're excited to explore are the potential for version
1:03:22 control in a local-first environment.
1:03:25 If you have, a layer like auto merge in your stack, which is really designed
1:03:29 around not just local-first data, but also storing, past history of
1:03:34 documents, what can you do with that potential to unlock for end users.
1:03:38 And another theme we're gonna explore, and have started exploring a bit already
1:03:43 is not just data in a local-first way, but code in a local-first way.
1:03:48 So if you put code on people's devices in a way that they can mod
1:03:53 themselves what does that unlock in terms of customizing our software?
1:03:58 This is sort of a fully maximalist local-first approach and it's something
1:04:03 that Peter and the lab have explored in the past but it's, you know, there's a
1:04:07 project called Pushpin, which I encourage people to look at if they haven't
1:04:11 heard of it, which is basically, an exploration of what happens if you have
1:04:15 local-first data plus flexible tools that can op that can operate on that data.
1:04:20 And I'm excited to keep pushing in those directions and see if we can basically
1:04:25 give people more control and empowerment over their computing experience on
1:04:29 top of this local-first data stack.
1:04:31 That is incredible.
1:04:33 I'm very tempted to also somehow contribute more in those various
1:04:38 topics that you've been mentioning.
1:04:40 But
1:04:41 My day has just so many hours, and I think I have still a lot of work ahead
1:04:46 of me with LiveStore and and Overtone.
1:04:49 However, I think there will be many problems that will have positive overlap.
1:04:53 I think we're still not done with the Cambria problem.
1:04:56 Maybe we get to collaborate there in the future.
1:04:58 We've been doing also some really interesting exploratory work around
1:05:03 what does it mean for an app to have version control for for users.
1:05:07 That is something I'm really interested in so I'm I'm sure
1:05:11 we'll we'll get another stab at collaborating on some project together.
1:05:16 But, yeah, Geoffrey, I've been looking forward to this for a long
1:05:19 time and really excited to share what we've been working on over the past
1:05:23 few years with a broader audience.
1:05:26 And thank you so much for for taking the time and sharing all of those
1:05:30 stories and and insights with us.
1:05:32 Thank you.
1:05:32 you
1:05:33 Thanks so much.
1:05:33 It's been fun.
1:05:35 Outro
1:05:35 Thank you for listening to the localfirst.fm podcast.
1:05:38 If you've enjoyed this episode and haven't done so already, please subscribe and
1:05:42 leave a review wherever you're listening.
1:05:44 Please also consider telling your friends about it, if you think they
1:05:47 could be interested in local-first.
1:05:50 Thank you again to Expo and Crab Nebula for supporting this podcast.
1:05:54 See you next time.