Its funny, when I was reading part time parliament paper (paxos) and while learning zig and odin language created a pure paxos library and then the first application for that library was to build a multi-master sqlite. The two approaches I took are very different in case of zig I build zaxonlite (https://github.com/insanai/zaxonlite) and have a benchmark on it that includes rqlite. It uses WAL replication as strategy. The second attempt I tried in odin is sqlodin (https://github.com/insanai/sqlodin), in this the system does not use WAL but run sql query on each node, as result subset of sql query in sqlite are not supported.
Having used D1 in production - it's not very reliable. There was a comment on HN from one of D1 devs [1] saying that D1 is a sql wrapper on top of durable objects that mimicks sqlite distributed like nature.
> I'll let you in on a sort of dirty secret:
It's almost always better to use Durable Objects storage, rather than D1. Even if you only want a single global database, it's better to implement that as a singleton Durable Object, than by using D1. Because that's all D1 itself actually is: a singleton Durable Object that exposes an API to its SQLite database. It's just a wrapper.
https://github.com/punnerud/mpedb is not there yet, focused on sync between sqlite3, Postgresql and Oracle. Shown that concurrent writes to sqlite3 is possible at the speed of Postgresql
(My own project)
Seems fair to me. The VC-backed SQLite services have been shouting about their services as if they came up with the idea, and this only balances that out.
He's right that histories start too late, and I'd like to see more posts like this that trace the history back further than the last 5 years. Like for databases, a lot can go back 20+ years and it's good to read how the idea came about (independently from many people), how it developed over time and why.
"Honesty" is what crossed the threshold for me. It's pretty weird to spend 12 years on something and not bother to write the thousand word history yourself.
I maintain one of the client libraries for rqlite - I’ve found it an excellent concept really well implemented. One of my favourite bits is it gives you back json making Dapper-like mapping super easy in .NET and I suspect other languages.
I know that latest rails suggests SQLite as one of the production options. I don’t get how it’d work with concurrent access though. Time to look into it.
Its funny, when I was reading part time parliament paper (paxos) and while learning zig and odin language created a pure paxos library and then the first application for that library was to build a multi-master sqlite. The two approaches I took are very different in case of zig I build zaxonlite (https://github.com/insanai/zaxonlite) and have a benchmark on it that includes rqlite. It uses WAL replication as strategy. The second attempt I tried in odin is sqlodin (https://github.com/insanai/sqlodin), in this the system does not use WAL but run sql query on each node, as result subset of sql query in sqlite are not supported.
Having used D1 in production - it's not very reliable. There was a comment on HN from one of D1 devs [1] saying that D1 is a sql wrapper on top of durable objects that mimicks sqlite distributed like nature.
> I'll let you in on a sort of dirty secret: It's almost always better to use Durable Objects storage, rather than D1. Even if you only want a single global database, it's better to implement that as a singleton Durable Object, than by using D1. Because that's all D1 itself actually is: a singleton Durable Object that exposes an API to its SQLite database. It's just a wrapper.
[1] https://news.ycombinator.com/item?id=48611834
Completely. D1 flaky as hell!
If you don't get any joy out of writing about your personal experiences in your own voice, why bother writing about them at all?
I've been looking for a reason to use rqlite, but so far sqlite + litestream have been safe enough (some dataloss acceptable in extreme cases)
My understanding is that rqlite gives you "stored in more than place" guarantees, which is nice if you need it. Much slower than a 1 node db though.
Is there a project that does all of these together?
- embeddable like sqlite
- clustered like dqlite
- has a standalone node like rqlite
- can be backed up to S3 like litesteeam
- optionally AP like cr-sqlite alongside CP
https://github.com/punnerud/mpedb is not there yet, focused on sync between sqlite3, Postgresql and Oracle. Shown that concurrent writes to sqlite3 is possible at the speed of Postgresql (My own project)
Was a fun read until the end where the author made damn sure that history records who drew the blueprint.
Seems fair to me. The VC-backed SQLite services have been shouting about their services as if they came up with the idea, and this only balances that out.
He's right that histories start too late, and I'd like to see more posts like this that trace the history back further than the last 5 years. Like for databases, a lot can go back 20+ years and it's good to read how the idea came about (independently from many people), how it developed over time and why.
"Honesty" is what crossed the threshold for me. It's pretty weird to spend 12 years on something and not bother to write the thousand word history yourself.
We use rqlite in production, it's the database we chose for (the control plane of) an open source observability solution we're building (morio.it).
Just came here to say two things:
- rqlite is amazing and warmly recommend - Philip is an exceptionally dedicated, kind, and helpful maintainer
5 stars, no notes. Thank you Philip, I'm eternally grateful for all your great work.
I maintain one of the client libraries for rqlite - I’ve found it an excellent concept really well implemented. One of my favourite bits is it gives you back json making Dapper-like mapping super easy in .NET and I suspect other languages.
I have never examined this in any detail before. It looks far easier than the mass majority of clustered database architectures.
https://rqlite.io/
using it for control layer pretty happy so far
can you query the local rqlite, or do you always have to go to the leader?
I know that latest rails suggests SQLite as one of the production options. I don’t get how it’d work with concurrent access though. Time to look into it.
Can use one sqlite per tenant or look into the new multi writer lib on github.
Wal or wal2 can help you with that.
There is also hctree from sqlite.
Very timely for me. I run dqlite in a small cluster and I'm looking to move off it to something more robust. What's recommended in this regime?
What kind of issues did you run into?
Loads of '500 database locked' errors. https://github.com/canonical/dqlite/issues/908
Like the idea of distributed SQLite, but do we have benchmark? What use cases it has advantage?