@marak in #lolashare

Currently digging into WebRTC DHT for Webtorrent

https://github.com/nazar-pc/webtorrent-dht
https://github.com/webtorrent/webtorrent/issues/288
https://github.com/webtorrent/webtorrent/issues/1152

cc @Matt @ev

@ev
Re: %uEx2pg8Vn

I'm going to read up on Webtorrent because I haven't spent a lot of time with the code.

I think keeping blob distribution in the browser is ideal, because we want to move ssb fully into the browser to eliminate our dependence on desktop and (hopefully) pub installs. AKA the holygrail of distributed awesomeness

@Matt in #lolashare
Re: %uEx2pg8Vn

@Marak

The DHT is only needed in the web browser. You should be able to use standard BitTorrent DHTs in an electron client. However, I couldn't seem to get this working. Not sure why.

One idea I had was to include a DHT (bittorrent-dht) in every pub. Then rather than trying to connect to a global bittorrent bootstrapping node, the client just tries to connect to the DHT of all known pubs.

However if you are going to be adding a DHT to pubs, might as well just use trackers. These already work in web browsers too. This is the current version works. The reason I was trying to move away from this was so that any standard ssb pub could be used.

Was thinking about turning every peer into a tracker so that you didn't need to be connected to the internet, it could work locally. See https://github.com/LolaShare/LolaShare/issues/45 --- work was started on this in https://github.com/LolaShare/LolaShare/tree/integrated-trackers

Really what you need to decide is:

Does it matter that you have to run a Ferment specific pub?

There already will need to be some specific plugins installed to handle seeding, and we already want to scope the network. When it was just me working on this, I decided this was too much of a personal commitment. I wanted to leverage the existing infrastructure. But since this time around we're focusing on this being quite separate from patchwork and the rest of ssb, it's probably fine.

With this in place, the current infrastructure is actually pretty good to go. The following could be improved:

  • all peers become trackers (so that it works offline)
  • try to connect to all known pubs/peers as trackers
  • try disabling webrtc based torrents, and just do plain old bittorrent (can do this with the webtorrent modules) --- this should help with CPU usage (however means that we unfortunately can't connect to web browser peers)
  • The current version loads everything into memory! Like THE ENTIRE DATABASE! This is not optimal as we grow. We should rebuild on top of patchcore (probably as a fork of Patchwork) which allows queries and streaming of datasubsets. Just need to replace the stylesheets, message types, and rebuild a few views.

Oh, another nice advantage of rebuilding on top of patchwork is that you get comments for free!


Sorry, a bit rambly, but hopefully helpful!

@marak in #lolashare
Re: %uEx2pg8Vn

Thanks for the information, this is making it a bit more clear.

I'm curious to what @Dominic thinks about leveraging existing infrastructure versus a separate network of pubs.

It might not be the worst idea to create a new network just for Lola. We could in theory merge them back in at a later date.