@dominic
{
  "type": "pull-request",
  "project": "%M0TrM+oJT2i/phUJO/fZ2wkK2AN2FB1xK0tqR7SNj58=.sha256",
  "repo": "%M0TrM+oJT2i/phUJO/fZ2wkK2AN2FB1xK0tqR7SNj58=.sha256",
  "text": "## use secure-scuttlebutt/flume\n\nthis refactors sbot to use the breaking changes in [%secure-scuttlebutt](%iljFzUwTYposC7vs2V6AZgObPqwRVNAXjxYVVUoG4tU=.sha256)/flume branch.\nunfortunately npm can't install `ssb://` git repos, so I couldn't put it the the package json.\nif you want to test, I recommend cloning secure-scuttlebutt and linking it into sbot.\n\n```\ncd node_modules/\nln  -s ../../secure-scuttlebutt # or whereever it is relative to here\n```\nAlso, you'll need to remove the indexes so they get rebuilt... `rm -rf ~/.ssb/db/*/*`\n(I should probably make flumeviews a bit smarter so they can do that automatically, or something)",
  "mentions": [
    {
      "link": "%iljFzUwTYposC7vs2V6AZgObPqwRVNAXjxYVVUoG4tU=.sha256",
      "rel": "mentions",
      "name": "%secure-scuttlebutt"
    }
  ],
  "branch": "master",
  "head_repo": "%M0TrM+oJT2i/phUJO/fZ2wkK2AN2FB1xK0tqR7SNj58=.sha256",
  "head_branch": "flume"
}
@ev
Re: %JBJRvToYT

@dominic Testing breaking changes now.

@ev
Re: %JBJRvToYT

@dominic unable to build


> secure-scuttlebutt@15.5.1 test /home/ev/secure-scuttlebot
> set -e; for t in test/*.js; do node $t; done

/tmp/test-ssb-feed
/home/ev/secure-scuttlebot/indexes/last.js:19
  var createIndex = Reduce(1, sfunction (acc, data) {
                                                  ^
SyntaxError: missing ) after argument list
    at Object.exports.runInThisContext (vm.js:78:16)
    at Module._compile (module.js:545:28)
    at Object.Module._extensions..js (module.js:582:10)
    at Module.load (module.js:490:32)
    at tryModuleLoad (module.js:449:12)
    at Function.Module._load (module.js:441:3)
    at Module.require (module.js:500:17)
    at require (internal/module.js:20:19)
    at module.exports (/home/ev/secure-scuttlebot/db.js:11:18)
    at module.exports (/home/ev/secure-scuttlebot/index.js:65:27)
@dominic
Re: %JBJRvToYT

sorry, try the latest

@ev
Re: %JBJRvToYT

secure-scuttlebot builds, but now I'm getting cannot find module graphreduce in scuttlebot. Probably not published to npm yet.

@dominic
Re: %JBJRvToYT

sorry, published

@ev
Re: %JBJRvToYT

@dominic Still getting

Registry returned 404 for GET on https://registry.npmjs.org/graphreduce

@Mikey
Re: %JBJRvToYT

got it to run after changing:

diff --git a/package.json b/package.json
index e8454fe..28b3c6b 100644
--- a/package.json
+++ b/package.json
@@ -14,7 +14,8 @@
     "cont": "~1.0.3",
     "deep-equal": "^1.0.1",
     "explain-error": "~1.0.1",
-    "flumeview-reduce": "^0.2.1",
+    "flumeview-reduce": "^1.0.1",
+    "graphreduce": "^2.0.0",
     "has-network": "0.0.0",
     "ip": "^0.3.3",
     "level-memview": "~0.0.0",

it somewhat worked, as in i could get patchbay to load once, but then i kept getting expected previous: ... found: ... log statements and browsing to git ssb web wouldn't load.

also i wonder if it's possible to have a migration script, since the re-indexing loses all the "message received" timestamps, which really messes with patchbay and kinda loses previous information, but also definitely only a nice-to-have.

anyways, this is cool, i really like the direction of flumedb.

@ev
Re: %JBJRvToYT

I decided to give this branch another shot after the call, but I'm getting this error and I'm not sure how to fix it:

/home/ev/sbot/index.js:65
          ssb.use(name, flumeview)
              ^

TypeError: ssb.use is not a function
    at EventEmitter._flumeUse (/home/ev/sbot/index.js:65:15)
    at EventEmitter.hooked [as _flumeUse] (/home/ev/sbot/node_modules/hoox/index.js:10:15)
    at Object.exports.init (/home/ev/sbot/plugins/friends.js:35:20)
    at /home/ev/sbot/node_modules/secret-stack/api.js:27:28
    at Array.forEach (native)
    at create (/home/ev/sbot/node_modules/secret-stack/api.js:26:20)
    at Object.<anonymous> (/home/ev/sbot/bin.js:57:16)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
@yetAnotherLiteCryptix
Re: %JBJRvToYT

@ev the way the plugins are uncommented is a bit clumsy, i think. trying to fix it to.

@yetAnotherLiteCryptix
Re: %JBJRvToYT

no.. wrong file, sorry. i'll get some sleep..

@dominic
Re: %JBJRvToYT

okay I have updated the flume branch to put the flume stuff beside the old database, not inside it, which means you can switch between them easily.

on this PR you need to simlink in secure-scuttlebutt on the flume branch.

@dominic pushed to %M0TrM+oJT...
@dominic
Re: %JBJRvToYT

Okay this version should now be runnable without obliterating your old version.
instead of loading the flume stuff within the .ssb/db directory, it will put it in .ssb/flume this means it won't clobber your old indexes, so you can switch back easily. Also, if it isn't up to date, it will copy your old data into flume's log, so messages will still be in the same order.

@ev
Re: %JBJRvToYT

Ok. @gb just tried this branch and got the

TypeError: ssb.use is not a function

error.

I double checked my supposed working flume, and I discovered that my big error was never pulling flume's branch. So I was actually just running scuttlebot@9.4.2

I'm not sure how to fix this, but we both don't have working flume versions.

@dominic If you try a clean flume install, does it work?

@Toady
Re: %JBJRvToYT

Thank's for trying, you two! That is also the error that I hit last time I tried but was to rushed to document it properly.

I also went through not using the secure-scuttlebutt flume fork on npm but that didn't help either... :{

@gb
Re: %JBJRvToYT

I got it working. Hmm.

@ev
Re: %JBJRvToYT

@gb Nah, that was you pulling off my branch where I pushed a version that wasn't flume. Neither of us have it working, it turns out. Both getting ssb.use is not a function.

@gb
Re: %JBJRvToYT

Is mercury in retrograde, brah?

@gb
Re: %JBJRvToYT

@dominic_temp today on the call you said you'd just pushed updates to flume -- am I looking at the right branch here or did I misunderstand you on the call?

@Toady
Re: %JBJRvToYT

Yea.. I wanted to voice this during the call yesterday (but two clients forbid me from speaking..).

I will try to document by steps more cleanly but 3 times I hit a will with this error.

@Toady
Re: %JBJRvToYT

cc @dominic_temp

@Toady
Re: %JBJRvToYT

ping @dominic

kinda sure I had the correct branch at some point.. :-/ trying again now.

@dominic pushed to %M0TrM+oJT...
@dominic
Re: %JBJRvToYT

@cryptix @ev @gb this should just be a clone and npm install now. It might take a little while as it moves everything across into the flume database though - if you want to see if something is happening, use sbot since to show where the various flume indexes are up to. If they all show the same number then the database should be good to go!

@ev
Re: %JBJRvToYT

@dominic I have a working flume! I had to restart flume a bunch of times because it kept blowing the stack, but eventually everything transitioned over to the new db.

@dominic pushed to %M0TrM+oJT...
@dominic pushed to %M0TrM+oJT...