tl;dr — you can now find remote categories and see your tracked/watched categories in /world.

A new alpha version of NodeBB was tagged today: v4.3.0-alpha.3. The biggest change is to the /world route, which up until now showed a list of topics from outside of the local NodeBB instance.

New to this alpha release:

1. A quick search widget was added, allowing you to directly search for remote categories. There is no need to navigate to to the search page to discover new categories.
2. Your list of tracked and watched categories will show at the top of the page.
   • "Tracking" and "Watching" categories—both local and remote—is how content discovery happens in NodeBB. Tracked categories will have new content show up in the "unread" page, while watched categories take that a step further and notify you when new content is posted.
   • Tracking and watching a category will tell NodeBB to subscribe to that remote community for updates

At this time we're continuing to look for stability issues with the remote category integration. We'll be working on QoL fixes as we move into the beta phase this/next week.

Happy Tuesday!

Today we've updated the NodeBB community forum onto the remote-categories testing branch, which means that users on the open social web that identify themselves as "Groups" will be rendered in NodeBB as categories. Prior to this, they looked like users.

Here are some examples of remote categories:

• Comic Strips (on lemmy.ml)
• Star Trek Social Club (on startrek.website)
• Social Web Foundation (a WordPress blog)

ActivityPub "groups" and forum categories have quite a few things in common — they don't usually post topic themselves, they "contain" topics, and they are usually administered by a separate group of users (moderators!) In many ways, these groups lend themselves to categories much more easily than they do as users.

Notes:

• We will likely be releasing this as v4.3.0-alpha this Wednesday. Probably this means you don't want this on a live forum just yet.
• A lot of the backend logic is complete, but a lot of the frontend UX will be worked on.
• You can "search" for categories (via "in categories" in the search page), paste the full handle in order to instruct NodeBB to pull a new category in.
• You can now no longer mention a remote category. Instead, create your topic right in that category itself. As it should be .
• Remote content coming in that is slotted into a remote category will still show up in your "world" feed. That is still intended to be where discovery of content outside the local NodeBB instance will take place.
• Report any bugs or confusing behaviours (and there will be some) here.

Screenshots

For awhile now we've hosted https://try.nodebb.org as our demo instance for people to see a plain NodeBB instance.

I have added a federated category on that instance now. You can of course still make test posts to this instance, but you can also feel free to fill @federated@try.nodebb.org with garbage instead

Please note that the "try" instance of NodeBB is reset every 24 hours, which means all follow relationships, content, etc. will only last at most that long. If you follow the category or an account on that instance, those categories/users will "forget" you followed them.

I've just merged the activitypub branch of NodeBB into develop, meaning that v4 is on-track to be the next release.

It also means that v3.x has entered feature freeze, and that only bug fixes will be merged/committed at this time.

Exciting stuff!

ActivityPub introduced to NodeBB the concept of sharing/announcing/boosting, and we enthusiastically supported this as a method of content discovery.

While we were processing these activities (and publishing our own), we weren't actively tracking them internally. That has changed in the latest commits (soon to be RC5); shared content from both local and remote users are tracked internally and can be viewed in the user profile.

For example, here is c/comicstrips from lemmy.world. As it is a Lemmy community, it only shares content, and does not produce any of its own.

Here is a quick recap of how sharing is handled in NodeBB:

• When remote users boost/repost content, NodeBB interprets this as a share.
• When local NodeBB admins move topics out of "uncategorized" and into a category, it is interpreted as a share and this activity is federated out to the admin's followers.

At this time unprivileged users cannot share content.

As reported initially by @kichae@lemmy.ca, there are some posts from Lemmy that didn't make it over to NodeBB. This turned out to be an issue with how content was parsed by NodeBB if source.content was also present.

The code has been updated so that if a content source of type text/markdown is provided, then that source is used instead of the generated HTML.

Outside of Lemmy, this also applies to any implementor (snac2 maybe?) that also implements the source.content property.

I'm happy to announce RC1 (that's Release Candidate 1!!) of NodeBB v4. Since the last v4 pre-release (beta 4), we've made additional bug fixes and stability improvements.

Here is the list of new features and fixes since the first beta:

• Emoji!
• Ability to search for a user by their handle directly from the search bar
• When encountering links to websites that advertise an ActivityPub alternate, NodeBB will try to load that site in NodeBB first
• Categories now have their own avatars instead of serving the generic site brand icon
• NodeBB will attempt to backfill an entire topic when encountering a new object, reducing the need to visit the original site to get the entire conversation. (This is contingent on the other server supporting resolvable context collections)
• Chats are now federating out as limited-visibility posts on the fediverse
• Improvements to the title generator
• Custom Fields now two-way federating (custom fields are a feature new to the 3.x branch as well!)
• Better compatibility with nodebb-plugin-feed

Timeline-wise this brings us close to the end of our funding from @nlnet@social.nlnet.nl for the 2024 calendar year. We've exceeded our own expectations of what we could do this year, and we've already applied for funding from NLNet again to really let NodeBB push the boundaries of what the ActivityPub protocol can do!

The RC signifies that the software has reached a (self-proclaimed) stable state, and broader usage is encouraged. If you have any questions, concerns, or trouble upgrading your existing NodeBB to v4, please call that out here for support.

There is now a soft feature freeze in place while we sort out the last remaining (hopefully none ) bugs, and we'll aim to launch v4 after that.

We have emoji!

I like emoji, I like them quite a bit. I grew up using emoticons (gosh, remember those?), and definitely love to sprinkle them in my forum responses.

If you've been following me since I've been contributing to the fediverse, you might've thought I eschewed emoji, but let's put that misconception to rest

Thanks to the hard work by @PitaJ, NodeBB now federates emoji used in posts outward to remote instances. This works across all emoji packs (android, apple, cubicopp, etc.) available to the emoji plugin, and also includes custom added emoji

This is a two-way integration, so custom emoji served by other instances are also picked up by NodeBB and displayed properly in posts.

You might notice that the emoji I use are all blobs. It's the default emoji set for all NodeBB instances, because blobs are amazing and it's a travesty Google decided to move away from them.

Long live the blobmoji

A little-known feature was added in the 3.x develop branch of NodeBB, custom fields. It was an often-asked-for feature that had a couple of plugins built to accommodate, but @baris decided to make this part of NodeBB core.

I will let him do a more detailed write-up on the NodeBB side, but suffice to say, those custom fields will now federate out with your profile:

Likewise, custom fields as defined by remote users will be saved and displayed in NodeBB in metadata boxes just like those defined in NodeBB.

In NodeBB, administrators are able to globally specify a set of fields that individual users can change/set.

Additional pages have been added to the NodeBB documentation regarding the following items:

1. Post visibility — how NodeBB handles incoming activities with public and non-public visibility, and how they are represented in NodeBB.
2. Conversational Contexts — how NodeBB builds out a conversational context when parsing incoming content.

tl;dr — you can now send me DMs. I blatently and unforgivingly abused the NodeBB chat system to make this work.

Ever since I started this project at the start of 2024, I knew that posts with limited visibility were going to be a sore spot. ActivityPub has the concept of "addressing", with the following valid entries:

• An actor (as:Person or similar) uri
• A followers collection (?!?!?!)
• The public collection

However, posts and topics in NodeBB have their privileges and access scoped to the category, which meant that while it is able to restrict visibility/posting/etc. to specific users, the system was not flexible enough to handle individual posts with different user visibilities.

Given that limitation, up until this week, if your object did not contain the public collection https://www.w3.org/ns/activitystreams#Public, it was automatically and unceremoniously dropped because there was no way NodeBB could display it to the targeted user without leaking it to other users.

I had gone through a couple iterations to try to figure out a way to make this work, but none stuck (see the follow-up technical post for more details), and I had just about relegated it to the "think on this for awhile" pile until a recent post by @scott@authorship.studio got me thinking outside of the box:

and your forums, direct messages, and inbox can have different feature sets.

Emphasis mine.

Unlike topics and posts, NodeBB's chat system is not constrained by the privilege system, and each chat room has its own collection of members, which in many ways made it a better fit!

I put together a proof-of-concept in a couple days, and we're test driving it now. So my apologies if in the past year you tried to DM me, and I didn't respond. I wasn't ignoring you, NodeBB just didn't know how to handle it, promise!

Support for accepting posts with limited visibility... coming soon...!

tl;dr — category actors in NodeBB now have better avatars/pictures! They're generated from the icon, color, and bgColor settings for each category.

Did you know that categories in NodeBB can also be followed on the social web/fediverse? Categories, as they are known here, are also actors on the fediverse, so you can find them just like any other account.

e.g. @general-discussion @support @nodebb-development @activitypub

@liaizon@social.wake.st opened an issue back in April (!!) pointing out that the avatar/picture served by NodeBB for category actors was not as good as it could be.

It's definitely possible to do something like this, although it is usually rather resource intensive. For example, when you paste a GitHub URL somewhere (e.g. Slack), it may try to unfurl the URL and display an image. GitHub blogged about how they did this using Puppeteer to generate the images.

I knew right away I didn't want to bundle what was basically an entire browser runtime into NodeBB's dependencies, so I needed a more lightweight solution.

I attempted to build my own SVG string and send that through as a data-uri, but not surprisingly, it was not handled by the implementations I tested.

Some more research this week led me to Vercel's OG Image Generation, which allows you to create images on the fly using Vercel Edge Functions. Since NodeBB is neither a Vercel app, nor do any people running NodeBB use Vercel, that ruled out this product.

However, more research into Vercel's implementation led me to Satori and resvg, a pair of programs that can generate an SVG from HTML, and a PNG from SVG.

From there, it was a fairly standard technical exercise to combine these with our font-awesome libraries in order to output our own implementation of this to generate colourful category avatars for its category actors.

For now, they are only generated on-demand when a category actor is requested, but if there are applications for the icons to be used elsewhere, then we could look into having them generated on direct file request.

Enjoy!

Today I tagged the v4.0.0-beta.2 release purely to fix a single issue — Update(Note) federation.

It turns out that just sending the Update(Note) activity was not enough, the underlying object needed to also have the updated field set in order to Mastodon to process the update. My guess is Mastodon checks that field and only processes the note if the value is greater than the last known value.

I'd write it off as a Mastodon-ism, but it's actually better to have that property present, so now it is.

Edit — does anyone know if Mastodon handles Update(Tombstone)? I sent one and Mastodon didn't do anything, which is technically ok. NodeBB handles it like a soft delete and removes it from public view.

If you experience issues related to the activitypub integration in v4.0.0-beta.1 or newer specifically, please let me know here so it can be triaged.

Last friday I quietly tagged a commit on the activitypub branch with v4.0.0-beta.1, which signals that the ActivityPub integration is now ready for beta testing.

For the most complete (yet readable) list of new functionality from the alpha, check out the "Road to Beta" project page.

tl;dr — some new features and a lot of fixes

• Editing a category now issues an Update(Actor)
• replies is now populated and responds with an OrderedCollection of direct replies (see the post about that)
• Proper sharedInbox support
• Better indicators for content that comes from non-local (aka "remote") users
• Conversational context synchronization mechanic (corresponding post for that)
• Moving a topic out of the "uncategorized" category will now federate out an Announce

By and large most incompatibilties have been resolved, although if you do find some issues, please do let me know in the corresponding bug report thread.

I'm looking to wrap up the year with some of the more difficult projects I've put off

• Object Integrity Proofs, which will also enable Inbox Forwarding
• Better handling of "Open in App" signals from third-party instances
• Post visibility support (or at least better handling so non-public messages aren'y unceremoniously dropped!)
• Ongoing integration with FEPs 400e and 7888

Now that the alpha version of NodeBB v4 has been released, it means that people are starting to look into using NodeBB for testing (or if they're really looking to have fun, production.)

The first step in enabling those intrepid individuals is providing documentation!

While a little bare-bones at the moment, I've written up a brief summary of what NodeBB brings to the ActivityPub landscape, as well as a run-down of various settings/toggles/switches, and how to switch over to the code branch that enables ActivityPub integration.

What I don't know is what's missing. There's a lot about NodeBB's integration that's very obvious to me, since I built it, but what don't you see that you'd like me to write about? Let me know.

Thanks!

For awhile I've been wondering what the slow-down was when I loaded /world, the curated timeline for a NodeBB user. Seemingly every time I figured out what it was, it would slow down again after some time.

Today it was taking 3+ seconds to load just the data, whereas our other pages (category listing, topics, etc.) all finish in under 250ms, including html generation, user data, etc... Certainly something was amiss!

As it turns out, asking your database to do an intersection of two data sets, one containing 23774 items, and the other containing 23606 items, is a little much. I guess this is what they mean when they talk about #webscale hah! (#fediscale?)

Specifically, I was intersecting the set containing all fediverse topics, with the set containing everything in the user's inbox, in their entirety. While this gave me a very precise answer, there was a huge cost to that precision.

After talking with @baris about this, we decided that sacrificing some level of precision would almost certainly go unnoticed. Instead of having the database intersect those two data sets, we would pull the most recent 500 entries from both and intersect them manually.

So, /world is now limited to 500 topics. Don't worry, you won't notice.

That simple fix brought the execution time down from ~1500ms to ~40ms, which is more in line with our expectations!

The latest commit to NodeBB's activitypub branch updates the post handler so that the replies property returns either an empty array (i.e. []) if it has no direct replies, or a resolvable URL to a replies collection.

This step completes the two-way traversal options for any given object. Given any object (e.g. an as:Note or otherwise), you can traverse up through parent objects via inReplyTo, and down through child objects via replies.

However, neither of these options are ideal for building out a reliable context around any given object. For a variety of reasons, somewhere along the parent or child chain you could run into issues retrieving the object (outages, timeouts, DNS, etc.), which is problematic when you're hoping to completely traverse a linked list.

What we need is to move past the uni-dimensional movement of inReplyTo/replies, and to the next dimension: the ability to see the entire context at once!

Luckily, one of the ForumWG's goals has been working towards a more reliable implementation using the rather aptly named context property, resolvable to an OrderedCollection. NodeBB already does provide this, Discourse is going to build out this capability, and I am interested in seeing if any other AP impementers are open to building out implementations against it.

Short-to-medium term, we're hoping to draft an FEP to codify this behaviour, but I'd love to see some working implementations first!