Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.
Hot on the heels (relatively speaking) of v1.1 of the Spec being released in November, v1.2 is now on the horizon! As a reminder, we're working towards quarterly releases of the spec going forwards - no hard dates yet though.
While this is certainly an improvement in speed, when it comes to writing software and updating implementations: quarterly spec updates may actually seem too fast. This is OK; implementations of the spec are not expected to update as soon as a new spec release is published. Rather, it is more realistic to expect that the ecosystem updates gradually over the course of the next few months/year after the release.
Happy Friday everyone! Today the Synapse team has released Synapse 1.50.0rc2. It fixes a particularly nasty federation-breaking regression that crept in during 1.50.0rc1. If you're currently running 1.50.0rc1, we implore you to update to 1.50.0rc2 as soon as possible!
In other news, a reminder that as of Synapse 1.50.0rc1, we have ended support for Python 3.6 and PostgreSQL 9.6 as per our dependency deprecation policy, as upstream has marked them as end-of-life.
We'll let the community test 1.50.0rc2 over the weekend to ensure that no other regressions have emerged (please test if you can)! If not, expect Synapse 1.50 to land sometime early next week.
Support CHGHOST caps (prevents leave+join on host change) ✅
Fix owner auto-registration (regression) 🐛
Upgrade to Mautrix 0.14 ⬆️
Sending RELAYMSGs have been requested for a while so now we do support that if the cap is added to request list. This makes plumbs on networks that support it work nicer. Receiving RELAYMSGs has not been implemented yet so they show up as external like before.
If you prefer keeping your IRC rooms clean without any IRC noise (mode changes etc.) you can now use a new FORWARD command in the network room to make all such events happen in the network room instead. This affects in-room commands as well.
ZNC users can now enjoy messages coming from your own clone in IRC rooms (including PMs!) if you wish so by keeping the znc.in/self-message cap enabled. This isn't extremely well tested yet but feedback welcome.
Not a breaking change but the caps support will cause connections to networks that ignore CAP requests take a few seconds longer unless you remove all the default caps for said network and it will never try requesting them again.
Mautrix 0.14 upgrade bumps the minimum version as well so packages beware.
We rewrote the whole settings page (as a stepping stone to 100% QML). Please test it and complain about everything I broke! We also tried to organize it a bit better so that similar settings are grouped together and tried to use the right controls for the right things. Best case you didn't notice anything. ;-)
And we also now show which profile a notification is for on KDE. I don't think other DEs have such a feature, so those will just show the generic Nheko as always.
We're about to release 0.2.23 after somewhat of a release hiatus. We've been working on two fronts:
Get the SDK out! Hydrogen was always meant to be easy to embedded, reuse in parts, and make customized versions of. But until now, actually doing that was quite challenging. Now that the SDK is out, you can use the hydrogen-view-sdk package in your projects and use parts of Hydrogen in your application. It's still early days, and we're still working out what symbols should be exported (finding a balance between APIs we want to support and utility), etc. Expect updates in the coming weeks. We're also not yet promising API stability for now, some APIs will very likely still change. Once we hit 1.0, we won't change things from underneath you anymore without increasing the major version.
Rich reply previews: as part of providing minimal support for threads (representing them as replies), we're switching from using the embedded reply fallback and actually looking up the replied-to message. One benefit is that reply previews will now updated when they are redacted (and edited, once we support that :).
Also fix some minor other things, like loading images when they are only partially visible, and a very basic location tile.
Issue with thread panel rendering timeline has been fixed
Reviewing plans for backward compatibility with clients that don’t support threaded messages
Polls
iOS and Android starting on the next phase of development, look out for poll editing and other exciting changes! You can enable polls in the Settings, under Labs.
You can now opt in to sending anonymous analytics data in the user settings on all platforms. Please enable it in the “Security & Privacy” menu, under “Analytics” to help us understand better how Element apps are used.
Mobile app users will see an opt-in screen on first startup on Android in 1.3.14 and iOS in 1.6.12
Maximised widgets merged into the release candidate and on track for the next release (see the Beyond Chat section for more info on this feature)
Fix code blocks being wrongly wrapped causing the line numbers to misalign, this was a regression in 1.9.8
Fixed ability to edit horizontal rules in markdown
Fixed some edge cases around Spaces not updating properly causing rooms to show up in the wrong Space
Add ability to cancel an outbound message during its encryption phase
Fix wrongly sending typing indicator when restoring a draft when changing room
Replace kick terminology with remove to be more inclusive
In labs (you can enable labs in Settings on develop.element.io or on Nightly)
Polishing bubbles layout
Upgrade to Electron 16 for Element Nightly - we are expecting this to help resolve some of the issues caused by Electron. Intending to release to production in 1.9.10 (two weeks away)
The maximised widgets feature in Element Web that Timo K. has been working on is now available for everyone on develop, and it's on track for the next web release. 🚀 It's a great match for widget-based collaborative editors, dashboards, games, and more. The widget becomes the focus of the room. You can optionally show chat from the room in a side panel, allowing easy discussion of the document / dashboard / game.
To try it out, add a widget to a room in the usual way, then look for the new "maximise" button in the room info panel's widget section. Please let us know in #beyond-chat:matrix.org or in issues if you have any feedback. 😄
Beeper is a universal chat app built on top of Matrix. We've created 12+ open source Matrix bridges and integrated them into an easy to use all-in-one service which does not require setting up your own homeserver. You can learn more at beeper.com.
Our team is growing! We’re now at 25 people, all remote around the world. Recent additions include hifi (creator of Heisenbridge) and Finn (creator of signald).
We are hiring many full-time remote roles including:
Bridge developers
iOS engineers
Product designers
Learn more here and apply through that site, or message @eric:beeper.com
Beeper Desktop
We recently released a Beeper Desktop update with a new room list and a ton of UI improvements. Check out the video below!
Made it easier to triage your inbox by moving unread dots to left
Made the list of connected bridges more compact
New link previews
Loads of bug fixes including improvements to scrolling
Most importantly, we launched a cloud Mac service that allows you to use iMessage with Beeper if you do not have access to an always-on Mac computer. Included with your Beeper subscription!
Voice messages are bridged in native format in both directions for all Beeper bridges
Signal bridge now supports disappearing messages
All Instagram message types are now bridged
Added support for Telegram message reactions
WhatsApp bridge now backfills 3 months worth of chats
Beeper Android
Coming soon: A recent addition to our Android team has added encrypted search by massaging seshat into the app. Expect an upstream PR for this into Element Android as well!
Trixnity version 1.0.0 is out! I decided to make it 1.0.0 not because it is out of beta, but because it has most features you need to build a usable client.
That means: cross signing has landed into Trixnity! The next big features will be room key backup and push notification.
Here is the changelog:
fully support cross signing
load members of rooms in an async way (you don't need to catch errors anymore)
reactive displayname and avatar url of the logged-in user
reactive avatar url for rooms
reactive is-direct-room state for rooms
better Android support for thumbnails
make part of device keys API public
merge SecureStore into Store (encrypt your database if you want to keep secrets secure)
move trixnity-client-api model-classes into separate module to use them e.g. for a matrix server implementation (thanks to @NicolasJouanin)
fixed long standing bug of wrong room name calculation
account data events are handled as if they have a key (like state events) to bypass inconsistency in the spec (maybe this will lead into a MSC)
Just cut a new release of the Ruby Matrix SDK, with 2.5.0 there's some preliminary support for Matrix 1.1 and the client/v3 API, the information for room knocking is exposed properly, some threading issues have had workarounds applied, and a bunch of fixes have been applied.
Currently running my own ping bot with it, I'm doing the GitHub (and hopefully also GitLab at some point) releasetracker bot, I've got my definitely not suitable for production MatrixFS, there's a notification module for TheForeman which we use at the university. And I've got some MSC hacks as well, like the MSC2108 testbed.
I just open sourced a library called Matrix-CRDT: https://github.com/yousefED/matrix-crdt - feedback very welcome! It allows you to use Matrix as a backend for decentralized, local-first collaborative apps. Above you see a collaborative rich text editor (like Google Docs) powered by Matrix!
You can try the Rich text editor here: https://bup9l.csb.app (see also the links in the repo, e.g. there is also a collaborative todo-list example)
If you have a distributed data structure and an algorithm that ensures all participants end up with the same result when their actions are combined, then that's effectively a CRDT as well. For Matrix, there a few research papers like https://arxiv.org/abs/2011.06488 which examine the CRDT-like properties.
First stable release of the gh-bot which is a simple webhook to Matrix bot made in Python.
As of now, the bot supports webhooks from Github, Gitlab and Gitea (although support for this one is very light).
Some other projects might do that way better than this bot but this is a learning project, to learn step by step how client interact with servers.
The next step would be to add a CRT.sh integration to get notified when a certificate is issued for a certain range of domains (which is something we want since we saw an IRC bot do the same).
Matrix-Art is a new social network prototype on Matrix.
It is a direct Devianart style clone. It currently has a focus on only images but is going to get extended to other media types eventually.
I am doing this as a toy project, so it may sometimes have slow progress. The goal is covering the main functions Devianart provides (Posting, Sharing, Profiles, Following, Collections, Comments) as well as integrations to other social networks on matrix using MSC3639.
A lot of things are currently still missing, but I am trying to get uploading working next :)
An instance is hosted at https://art.midnightthoughts.space/ and the code is at https://github.com/MTRNord/matrix-art
Currently, the interface is limited to viewing basic data from a matrix room. In the future, there are plans to have an open registration as well as login with any account to use it. :)
Note however that I suggest against using your personal account just yet, even though the login works, as it may have unexpected room joins at this time. This is known and expected, but not production ready.
Also, at this time, there are no plans of e2ee support. This may get added after the main features are finished.
I just wrote an article about my experience using Matrix, and the question... is it worth posting in the news? The article is written in Russian and published on Habr... and since it's pretty big I'm not sure about translating it to English 😅
Here is the link: https://habr.com/ru/post/599777/ - it also mentions TWIM.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
What a pleasure it is to be back with the community for the new year! This week Erlend is detailing what Commune is, and I'm flabbergasted by how well thought-through the project is.
Dept of we've been away for holidays
This week in Matrix should be called Three Weeks in Matrix, since there hasn't been TWIM updates during the holiday season. Nico has published a first and second communitwim while I was away. All the news reports since the last official TWIM still made it to the post you're currently reading!
Your regular spec person, anoa, is out today so you're stuck with me, not-anoa. This time I got the script to work though 😇
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
MSC Status
Note: This is for the last 3 weeks given TWIM holiday :)
In terms of Spec Core Team MSC focus for this week the last little bit, we've been working on getting MSCs merged into the formal spec itself in preparation for v1.2 this quarter. We expect the release to happen quite soon and to contain Spaces, room versions 8 and 9, and refresh tokens (no spec PR yet) in addition to all the other wonderful stuff which has landed.
The merged MSCs can be seen over at https://spec.matrix.org/unstable/ where they're queued up for the next release. Look out for Added in v1.2 labels throughout the spec, denoting what is new and exciting.
Random MSC of the week
The random MSC this week is MSC2192: Inline widgets (I promise it was random, after a few re-rolls). Inline widgets are a concept that allows for rich functionality in the timeline without needing to necessarily specify an explicit event type. For example, video embeds, minigames, etc could all be represented by inline widgets instead of dedicated event types.
The MSC needs updating to handle MSC1767: Extensible Events (and friends), but once there it could be a very powerful bit of functionality, with free fallback thanks to Extensible Events.
Sliding sync (aka sync v3) is described in MSC3575 - it's still very early days for the proposal which means a lot can change: and YOU can be a part of that. Please take a look at the MSC and provide any and all feedback, be it on the names of keys, format of JSON objects, or the kinds of operations that can be performed. I'm particularly interested in feedback from client developers who have complex room list sort orders or room list filtering requirements and bot/bridge developers who typically don't have a visible room list UI. Any and all feedback at this stage is welcome, visit #sliding-sync:matrix.org to join the discussion.
We're back! New year, new release candidate: we published Synapse 1.50.0rc1 today, marking the end of life for our Python 3.6 and PostgreSQL 9.6 support. We've also extracted some shared utilities into their own package, matrix-common, which is used by Synapse, Sygnal, and Sydent.
We'll talk more about what's in 1.50 next week when it formally releases.
v1.2.9v1.2.10 got released as a maintenance update. With early support for Matrix 1.1, S3 storage classes, and blurhash fixes it's worth the upgrade though there are other goodies - check out the changelog, and report bugs to the issue tracker 🙂
And as a end-of-year update, my Helm Charts have gotten updated yet again. With element-web ending up on 1.9.8, matrix-synapse on 1.49.2 (and .1 before that), and matrix-media-repo on 1.2.10
The first iteration of a Dendrite Helm Chart has been released under the k8s-at-home charts repository. It currently supports a full monolithic deployment and requires minimal configuration to get up and running (just need to generate a matrix key and mount it as per the instructions).
If polylith is your thing then I recommend the chart by S7evinK for now, although it is likely these charts will merge in the future.
An add-on for the matrix-appservice-webhooks bridge. Webhooks are essentially web interfaces for applications to "push" data to. The bridge can receive messages in a certain format, which is nice if the notifying app can be configured. Often it cannot.
Advanced Templating! It is now possible to set format and msgtype based on arbitrary values from the webhook JSON via Jinja2. Shoutout to qg for suggesting this and giving their feedback!
compatibility with matrix-appservice-webhooks forks that read avatarUrl instead of avatar_url
allow mxc:// URL avatars
improvements to the GUI, including automatically resizing text areas and msc:// avatar URL preview
improvements to templates/examples including making use of above features
Howdy folks, it is the time of the year where everyone scarpers! Anyway, perfect time to announce that matrix-hookshot has gotten it's first major release!. 1.0.0 is here!
For those not in the know, the hookshot bridge is used to bridge GitHub, GitLab, JIRA and Generic Webhooks into Matrix rooms. It doesn't just bridge into existing rooms, but can also spawn dynamic rooms based on aliases, send you your notifications in a DM and do lots of other wonderful things!
The notable changes from the 0.1.0 release are:
The bridge has now been renamed from matrix-github to matrix-hookshot.
Now supports JIRA and Generic Webhooks in addition to GitHub and GitLab.
You can get involved and start playing with it by checking out the release here
And that will be my last TWIM entry of the year. Have a good one and stay safe all 🐶
That was in 2021! Half-Shot is back in 2022 with more news
Hey folks! It was only last year we released the 1.0 release of hookshot, after many years of work to get it that far. I'm happy to announce that this week we've got another release. There are a number of buxfixes and improved documentation pieces landing (special shoutout to HarHarLinks for ensuring the docs are competent). The highlights are as follows:
Added support for Figma webhooks. this also means the archival of my old project matrix-figma
Support GitLab wiki page change events.
Added a new script validate-config which allows you to check your config file for simple errors. Handy for people writing ansible roles!
Add support for a html key on generic webhooks to set the HTML content of a Matrix message.
The project can be found over at https://github.com/Half-Shot/matrix-hookshot/, with pretty pretty docs at https://half-shot.github.io/matrix-hookshot. We're also in #hookshot:half-shot.uk if you prefer using Matrix to learn about these things!
UNPLUMB network command to force unplumbing without being in the room
Proper SASL external with CertFP with mechanism override option (see notes)
Disconnect and cleanup from networks that have no rooms open ♻️
Reply (and reject) DM requests to ghosts with QUERY command ↪️
Try to keep IRC users in the room at all costs if they are on the IRC channel
Fix assumption of all IRC replies to have arguments
Prevent accidental namespace changes to cause mayhem
Finally convert from homegrown Matrix API stuff to Mautrix
Bump Mautrix requirement to 0.12=>0.14
Conduit support was broken in 1.8.x but fixed again in 1.9.0, sorry
Finally there's network level spaces support with a new SPACE command. This creates a new bridge controlled space for the network and automatically manages rooms in and out. There's an issue/feature with Element that all rooms that have been converted to DMs with /converttodm will appear in all bridge spaces. The workaround is to convert them back to regular rooms.
CertFP SASL has been updated to do SASL external flow by default. If you are upgrading and have used CertFP with OFTC you need to run SASL --mechanism=none for it to connect again.
Abandoned networks where the user has left all rooms including the network room will now automatically disconnect and cleanup. This is more in line what people would expect and prevents idle connections from hanging around.
This is a small bug fix release. If you reported an issue, there is a 15% chance it is fixed now! This release also supports pinned messages, although those will only show up after someone changed the pinned messages in a room currently (we didn't want to force a full resync just for such a small feature). The spaces list is also now nested, Nheko offers you quick access to your recently used reactions and Nheko will show you your direct chats in the sidebar. Apart from that there are quite a few bugfixes and smaller improvements, you can find the full changelog and downloads here: https://github.com/Nheko-Reborn/nheko/releases/tag/v0.9.1
Thank you everyone, who helped shape this release!
Nheko now is a lot more efficient. We now use one Threadpool instead of 3, got rid of more than 60% of the allocations when scrolling through messages, layout half as much content when scrolling, blurhashes decode in 10% of the time and jdenticons allocate ~10% as much temporary buffers. We also deleted around 1000 lines of unused code. Tooltips also shouldn't steal the mouse focus when scrolling anymore, which could lead to sudden stalls when scrolling.
Additionally edits now replace existing notifications, tastytea added a manpage and fixed blurry or incorrectly sized custom emoji, you can now send custom emotes via the inline completer using ~ and completers now show a scrollable list with more Elements than before. Advanced users can also now opt into an insecure client side secrets storage via a hidden setting.
We're baaaaack! Anyway, last week Drake just translated all of Nheko into Spanish, you can now zoom in and pan in the image viewer, emojis shouldn't split up into their segments anymore and Nheko should always be sending the qualified version of it (according the to the unicode test files, not by just appending FE0F). Blurhashes should be even faster still and we now have support for running the call event loop on macOS and Windows (although call support is still disabled there for now).
We are also now working to restructure our README. It has a lot of outdated stuff in it and you really want to see screenshots pretty early in the README! You can sneak a peak here: https://nheko.im/nheko-reborn/nheko/-/tree/README_updates
This week the Threads team has focussed their efforts on improving some of the smaller details and pesky bugs we’ve found in our first rounds of testing.
If you’d like to help us test Threads we’ll be asking for help in the Community Testing room over the next few weeks. Join#element-community-testing:matrix.org to stay in the loop.
Polls
Polls is available in Labs on all clients! While we know there are some minor improvements to be made, we’re proud of where we are and would love for you to start using it!
Community Testing
Our next session will be on Wednesday, 12th January at 16:00 UTC (17:00 CET). We will be testing the new release candidates on all three platforms! Join us at#element-community-testing:matrix.org
Work continues on the integration of PostHog analytics. With your explicit permissions we’ll receive anonymous usage data that will allow us to understand the areas of Element that are helpful (or not). This info will help fuel things like our Information Architecture project.
In labs (you can enable labs in settings on develop.element.io or on Nightly)
Information Architecture improvements are still being worked on - try it out by enabling things like the new Spotlight search and Breadcrumbs.
Message Bubbles are improving; We’ve been hard at work preparing them for a release in the coming weeks.
During the Christmas break we increase the speed of the app launch by 7x! Once stabilised this update will land with you.
Analytics changes have been merged into the RC and opt-in will be available soon.
In development:
More Spaces improvements are underway; You’ll soon be able to add rooms to Spaces, update room settings, and have room interactions on the ‘long-press’.
Improved onboarding! We’re hoping to make the first few steps in Element easy by simplifying some of the first tasks users take in the app, including signing up.
We’ve started building Message Bubbles on iOS. This will help to distinguish the messages you send from the messages you receive.
We now implement MSC2574! This MSC proposes a standard format for marking up resources (PDFs, other document formats, audiovisual media, websites, geospatial data...) using Matrix.
Room avatars are now displayed in the welcome view for PDF rooms
There's now a UI for using spaces to manage and share PDF collections
Reactions now work like element-web: click to mirror an existing reaction or redact your previous reaction
Next, I'm aiming to implement MSC3592, which proposes a standard format for basic markup on PDFs. Stay tuned! And if you're interested in learning more, come visit #opentower:matrix.org.
Matrix Wrench is a web client to tweak Matrix rooms. After formerly calling it Matrix Navigator or Matrix Screwdriver, I finally settled on the name Matrix Wrench. ¯\_(ツ)_/¯
Version v0.2.0 comes with a Network Log which displays curl equivalents for all network requests. It also allows to easily add and remove room aliases.
https://gitlab.com/jaller94/matrix-wrench/
Cactus Comments is a comment system for the open web, built on Matrix.
We released version v0.11.0 of the client!
The client has been relicensed to LGPL. This means that you can now use Cactus Comments in non-GPL compatible projects.
But mainly, this release brings a bunch of CSS improvements: introducing automatic dark mode, and making it easier to change colors with CSS variables.
Here's the changelog:
Relicense from GPLv3 to LGPLv3.
Rewrite large parts of the stylesheet to use flexbox.
Introduce CSS variables to the stylesheet.
.dark and .light CSS classes with default values for dark/light mode.
Bugfix: "View More" button no longer blinks when auto-refreshing short comment sections.
matrix-bot-sdk has had a v0.6.0-beta.3 release with beta support for crypto! It even includes documentation!
The crypto is considered beta quality at the moment: good enough to use for somewhat unimportant bots, but not fully recommended for production just yet. With that being said, I'm interested in bugs you run into - please use the issue tracker if you run into crypto not working.
Tutorials for the crypto setup are at https://turt2live.github.io/matrix-bot-sdk/tutorial-encryption.html
Note for appservice support to work then you'll need a Synapse with these PRs enabled (may require manual merge too):
https://github.com/matrix-org/synapse/pull/11538
https://github.com/matrix-org/synapse/pull/11617
https://github.com/matrix-org/synapse/pull/11215
For non-linux platforms, the rust-sdk will try to build itself which means you might need a working Rust stack. The Rust SDK repo itself has more information:
Bots and appservices don't automatically support encryption, but adding encryption should be easy. The Rust SDK dependency is required in either case, sorry.
Complement has seem some updates this week in the test output that is produced onto the CLI. Details on how to add this to your CI process are contained in the README. Here's the difference when viewed using Github Actions:
Polyjuice Newt, the newest addition to the Polyjuice project, is an Elixir binding for vodozemac, the new Olm/Megolm implementation in Rust. At the time of writing, Polyjuice Newt supports encryption and decryption using Olm and Megolm, but by the time you read this next year, it may also support the SAS verification functions and pickling/unpickling.
I don't think we've previously had any Nix/NixOS/nixpkgs related entries in TWIM, so I'll start ^^
The current unstable channel has extended its Matrix ecosystem support to also include Heisenbridge and Conduit packages and modules. This makes it super easy to deploy any of those services: For example, my configuration for Heisenbridge is 21 lines long, and Conduit is only 11 lines. You can browse the available configuration options online: services.matrix-conduit, services.heisenbridge (note that some of them are freeform and simply forward to the upstream configuration).
For those that are not into NixOS, a module is the code that turns the declarative configuration files into your running system setup. As an example, if you enable services.heisenbridge the following things are done for you:
Create a new heisenbridge user and group for the service
Create and manage the registration file for the homeserver (i.e. automatically regenerate it after the configuration changed)
Create a systemd service that runs the heisenbridge command with the requested bridge configuration. The unit also sets a few systemd security hardening options.
Jip J. Dekker did the initial work of adding Dendrite support to the playbook back in January 2021. Lots of work (and time) later, Dendrite support is finally ready for testing.
The playbook was previously quite Synapse-centric, but can now accommodate multiple homeserver implementations, with Synapse still remaining the default.
A new Matrix guide has come into town: https://joinmatrix.org
In the hopes to expand Matrix's reach to the non-technical population, this guide is intended to give quick directions on how to use Matrix, as well as clear comparison between Matrix and other dominant platforms.
The pages are available on GitHub. Open to contributions!
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server.
If you’re reading this - congratulations; you made it through another year :) Every winter we sit down and review Matrix’s progress over the last twelve months, and look forward to the next - for it’s all too easy to get lost in the day-to-day development and fail to realise how much the overall project is evolving, especially when it’s one as large and ambitious as Matrix!
Looking back at 2021, it’s unbelievable how much stuff has been going on in the core team (as you can tell by the length of this post - sorry!). There’s been a really interesting mix of activity too - between massive improvements to the core functionality and baseline features that Matrix provides, and also major breakthroughs on next generation work. But first, let’s check out what’s been happening in the wider ecosystem…
The Matrix Ecosystem
Over 2021 the Matrix ecosystem has expanded unrecognisably. This time last year we were aware of 2 governments who were seriously adopting Matrix at scale (France and Germany), with the UK and US starting to roll out initial deployments. 12 months later, and we are now aware of 12 governments who are adopting Matrix in various capacities - and we hope to be able to talk about at least some of them in public in 2022! The UK and US have both progressed significantly too.
Meanwhile, one of the most exciting new public sector stories this year has been gematik: Germany’s national healthcare agency, who announced Matrix as the basis for interoperable secure messaging throughout the whole healthcare sector. This is a genuine step change for Matrix: rather than a government putting out tenders for “a secure messaging solution”, instead we are seeing tenders for Matrix solution providers. The Matrix industry is real; it exists today, and we’re seeing more and more new providers such as Famedly (building on the Flutter/Dart stack which powers FluffyChat) and Folivonet (building on the Trixnity Kotlin Multiplatform stack) stepping up to get involved - as well as many more big incumbents. We created Matrix in order to bootstrap a new decentralised communication industry, and frankly it’s amazing to see it actually taking shape.
Another big step change has been the number of existing chat providers looking to become part of the wider Matrix network. Back in September our friends at Rocket.Chat announced that they’re working on Matrix support for federation, perhaps inspired by our case study in making Gitter speak Matrix - and meanwhile Matrix comes up a lot in the context of Twitter’s Bluesky initiative, and a few big players we can’t yet mention have also been in touch wanting to natively talk Matrix too.
We’ve also seen a huge shift in big enterprises adopting Matrix for self-sovereign secure communication (although we can’t drop any names yet 😔). This may have been spurred on by such misadventures as Electronic Arts being compromised via a leaked Slack access token, but it feels like many of the biggest organisations now realise that unquestioningly handing their data to Slack or Teams is a bad idea, when they could have an end-to-end encrypted deployment of their own instead.
There has also been a turning point in legislation in favour of Matrix - with the EU Digital Markets Act pushing hard for interoperability for ‘big tech’ communication services in the EU (see Amandine’s take here), and meanwhile Eric Migicovsky, CEO at Beeper has been busy testifying to US Congress on the merits of interoperability too. It’s not inconceivable that we will soon live in a world where governments mandate that the walled gardens will finally have to open up, and we may see a whole new level of interest in communication providers wanting to join Matrix!
Communities themselves have also been embracing Matrix more and more over the last year: we were incredibly proud to host FOSDEM 2021, the world’s biggest open source conference via Matrix (all 35K attendees!) - and we’re gearing up to do it again in February for FOSDEM 2022 (this time with our very first FOSDEM Matrix dev room!). We were also really glad that Libera.chat let us point a dedicated homeserver and IRC bridge at their new IRC network (meaning you can join anywhere on Libera from Matrix as #channel:libera.chat, and talk to anyone as @nick:libera.chat). High profile open source projects have been adopting Matrix all over the place - Debian, Fedora, NixOS, Arch, Tor, Ansible, WHATWG and many others (check out this list!) now have their own Matrix servers and spaces. (You know things are busy when we haven’t had time to do a big blog post to announce folk as important as these joining the network!)
Finally, there has been an explosion of new projects and milestones in the wider community - Conduit entered beta as a super exciting lightweight Rust homeserver implementation; FluffyChat hit 1.0 with an impressively polished Flutter-based experience; Beeper pre-launched to huge amounts of mainstream excitement; Cinny exploded out of the blue as an incredibly elegant next-generation web client; Keanu materialised from The Guardian Project as their glossy Matrix client suite; Commune appeared as a hybrid messageboard/chatroom interface; Nheko has matured significantly with huge E2EE improvements and feature and VoIP polish; NeoChat and libQuotient development is progressing solidly; Fractal is busy with the fractal-next rewrite to move everything over to matrix-rust-sdk and GTK 4; Syphon continues to forge ahead as a privacy-focused Flutter-based client, and non-chat clients like The Board, Populus and Matrix Highlight have started to appear in earnest too! We also had a super successful Google Summer of Code this year, with a record number of 7 students participating in both core team and community projects.
Please note this is just a random sample of all the community news over the last year - to get more colour on what’s been going on, we highly recommend flipping through the This Week In Matrix archives!
The Matrix Spec
The Matrix spec is the single source of truth of what Matrix actually is, and this year it got some major improvements thanks to a beautiful new website at https://spec.matrix.org thanks to Will Bamberg, formerly of MDN (and who’s now back fighting the good fight with the MDN team at OWD).
Aside from the new spec site, we also released our first official point release in a while - Matrix 1.1, and we’re going to aim to keep regular releases happening once a quarter from here on in. It’s also worth noting that it’s very much a feature and not a bug that spec releases lag behind the various spec proposals which fly around as the core team and community experiment with new features like spaces, threads, etc. We very deliberately only merge change proposals to the spec which have been proven to work in real life implementations, and which have fully passed the spec review process (along with any dependencies they might have!).
Talking of which, in 2021 we saw a record 109 Matrix Spec Change proposals (MSCs) created. Even better, we closed 62 MSCs - so while the backlog is still growing, we’re still making very concrete progress. Of the 109 new MSCs: 34 were from the wider Matrix community, 34 were from ex-community contributors who are now part of the core team, 13 were from the founding Matrix team, and 28 were from folks hired to work on Matrix by Element on behalf of the Matrix.org Foundation. This feels like a pretty healthy blend of contributions, and while it’s true that spec work could always progress faster, things do seem to be heading in the right direction.
In the new year, the Spec Core Team (responsible for reviewing MSCs and voting on what gets merged to the spec) is going to make a concerted effort to carve out more dedicated time for spec work - thankfully one of the side-effects of Matrix growing is that there are now a lot more people around with whom we can share other work, hopefully meaning that we can put significantly more hours into keeping the spec growing healthily.
Synapse
Synapse is the primary homeserver implementation published by the Matrix core team, and its maturity is unrecognisable from where we were a year ago. One of the big breakthroughs has been stabilising memory usage through caching improvements - the Matrix.org synapse now reliably only uses 2-3GB of RAM on its main process, despite its activity having more than doubled over the last year (up from 513K monthly active users to 1.11M!).
Further signs of maturity include Synapse’s radically improved new documentation and the new module API, the fact that mypy type-safety coverage has improved from ~75% to over 89.7% (across 151,903 lines of code!), and the fact that Open Tracing support has matured to the point that visualising complex cross-worker behaviour is nowadays a genuine pleasure. Frankly, Synapse should be feeling robust and stable these days - if you see folks claiming otherwise, please check they’re not basing that on outdated info (or failing that, get them to file bug reports that we can jump on!).
Meanwhile, on the feature side, we’ve landed a huge spate of long-awaited core functionality. Probably the best way to track it is by the Matrix Spec Change proposals (MSCs) which have been implemented (although I dare you to also go and check out Synapse’s changelog, all 675KB of it, which is frankly a thing of beauty and will take you down a rabbithole all the way back to v0.0.0 in Aug 2014 if you so desire ;P). Major MSCs which we’ve landed include:
Spaces! It’s hard to overstate how positive this has been for Matrix’s usability: at last, we can group our rooms together however we please, both for our own edification and to share with others - and we can view space hierarchies over federation, complete with pagination (MSC2946) as well as specify who can join a room based on whether they’re a member of a given space/room (MSC3083).
Threads! Yes, that’s right - coming any day now to a Matrix client close to you, we have ‘classic’ threaded messaging landing, providing sidebars of conversation through the new m.thread relation type (MSC3440), building on Matrix’s existing aggregation API as used for edits and reactions. We’ve chosen to prioritise single-level-deep-threads rather than arbitrarily-deep-trees (MSC2836) as it maps more easily to a chat UX, although the two approaches are not mutually exclusive.
Aggregations! Everyone’s favourite bête noire in Matrix tends to be that aggregations for edits & reactions predate today’s Matrix Spec Change process and went mainstream without using a vendor prefix before their spec had been stabilised. Better late than never, we’ve taken advantage of Threads to go back and fix what once went wrong - and now MSC2674 and MSC2675 and friends are hopefully on a much better track to provide a basis for how aggregations work - both in the spec and in the reference implementation in Synapse.
Social Login via multiple SSO providers (MSC2858) - almost 50% of new registrations on the Matrix.org homeserver now use social login! Interestingly the split of SSO usage is roughly 70% Google, 12% GitHub, 11% Apple, 6% Facebook and 1% GitLab. Make of that what you will!
Knocking (MSC2403)! Huge thanks to Sorunome and Anoa, we now support the ability to knock to ask to join a room if not yet invited. If this sounds unfamiliar, it may be because it hasn’t landed in Element yet, but expect it to land next year.
Refresh tokens (MSC2918)! At last, we have a standard way for clients to refresh their access tokens, so that if your access token leaks it will not give access to your account indefinitely. (This also has yet to land in Element, but has been proved on a branch on Hydrogen).
Finally, last but not least, Eric from Gitter has been fearlessly hacking his way through some of Matrix’s gnarliest problems in his quest to bring Matrix+Element up to full feature parity with Gitter. In practice, this means adding the ability to incrementally import old history into existing Matrix rooms (MSC2716), so we can expose the vast amounts of knowledge in Gitter’s archives directly into Matrix - and in future provide bridging in general of existing archives (Slack, Discord, mailing lists, newsgroups, forums, etc.) into Matrix.
This is a tough problem, as Matrix rooms are fundamentally immutable - events sent into a room cannot be changed. However, we can bend time a bit and add old chapters of history to the room as if we’d just discovered them down the back of the sofa - and this is what MSC2716 does. The (rewritten!) spec proposal is a thing of beauty and well worth a look, and you can see an early preview in action back on Matrix Live in June. Over the last few months it’s been merging and maturing in Synapse and we should see it in the wild in the near future! And for bonus points Eric’s also just added in Jump-to-date support (MSC3030), letting clients jump around room history by timestamp - another Gitter feature that we sorely need, and will also help us publish excellent Gitter-style online chat archives in future. You can see it in action in last week’s Matrix Live!
Element
Meanwhile, on the client side, Element continues to act as a flagship client to drive the development of the official client SDKs we ship as the Matrix.org Foundation - and our focus more than ever before has been to ensure that Matrix can be used to create mainstream-usable polished glossy apps. After all, Matrix will only succeed if clients emerge which can punch their weight against the enormous incumbents - be they Slack, Teams, WhatsApp or Discord.
This year, improving UX quality has been front and center - and hopefully the shift has been obvious in the app (and huge thanks to everyone who tweeted/tooted/enthused about improvements when they saw them!). Part of this has been ensuring that all new features are built in a design- and product-led fashion by folks who are explicitly focused on product engineering - with product design involved from the outset and with coordination and focus provided by product management folks. This is far from the typical way that FOSS operates, but if we’re to succeed against the incumbents we have to beat them at their own game (just as, for instance, Mozilla wields conventional product management in their browser wars).
More recently, there’s also been a major shift towards structured user testing in order to evaluate new features and analyse how users trip over the app in general, including radically improved analytics (for those who opt in!) to help visualise which bits of the app aren’t working. In the new year, the expectation is to double down on user testing: quite simply, if you can hand Element to a casual mainstream user and they can get the core jobs done (sign up, chat to someone, call someone, etc.) without tripping over, then mission successful :)
The Element blog covers the work this year from the Element side, but from the Matrix side, the key changes include:
finalising Spaces as a way to group together rooms - providing the equivalent of Discord servers or Slack workspaces, or alternatively letting you gather your own rooms together into a private space.
building out Threads (available in labs; launching soon!)
Social Login!
radically improving Element’s Information Architecture (i.e. the layout of the UI, so that the panels and buttons are correctly semantically grouped together in a visual hierarchy)
adding Location Share (available in labs; launching soon!) powered by MSC3488 (and in future MSC3489 for live-location sharing - in dev on iOS right now!)
adding Chat Export, thanks to the amazing GSOC work by Jaiwanth
From a spec perspective, it’s been particularly exciting to be finally using Extensible Events (MSC1767) for many of these new features: voice messages, location sharing and polls are all experimenting with this new idiom for expressing richer structured data over Matrix while presenting a consistent and useful ‘fallback’ representation for clients which don’t know how to natively render the richer data.
We’ve also done a huge amount of work this year in improving 1:1 VoIP - both via MSC2746 and within the JS, iOS & Android Matrix SDKs. If you haven’t tried doing a 1:1 call via Matrix recently we’d highly recommend giving it a go - probably the main remaining bug at this point is that we need to find a better default ringtone for Element(!). Huge thanks go to Šimon Brandner both for his community contributions to VoIP and across all of Element Web - including proper screensharing for 1:1 (and group!) VoIP calls. This has also laid excellent groundwork for native Group VoIP/Video over Matrix - more on that later.
On Element Mobile, work on all the above features has been balanced by fighting against the various platform’s quirks, and lots of under-the-hood work improving performance. iOS has gone through a long journey to get back to stability after iOS’s push notification API changes, while also improving incremental sync performance by rearchitecting the local cache in the client. Android meanwhile has been working away improving the app; reworking Notifications, migrating to Kotlin coroutines and Hilt, and closing over 690 GH issues. Android has also had its fair share of dramas, including some recent long Play Store review times, but we’ve come through the other side intact.
However, we’ve been thinking more and more about the nightmarish pain point that is the amount of time we spend implementing the same features across the three different platforms. This becomes particularly apparent for security-sensitive features such as end-to-end encryption, or major API changes such as aggregations, spaces or sync v3 (more on that later). Or simply rapidly sharing improvements to implementation best practices between platforms.
Historically we consciously built platform-native Matrix SDKs in order to provide entirely idiomatic SDKs for other Matrix developers to use - and also to better dogfood the protocol and ensure that the heterogenous implementations could interoperate successfully. However, in practice, relatively few third party projects other than Element build on top of matrix-ios-sdk and matrix-android-sdk2 - and meanwhile there are more than enough other Matrix clients out there nowadays to dogfood interoperability against (including alternative experimental clients from the core team such as Hydrogen).
So, we’ve been thinking increasingly seriously about how to solve this…
A new hope: matrix-rust-sdk
matrix-rust-sdk is an attempt to build a new reference client SDK for Matrix which can be used by as many platforms as possible - hopefully forever stopping us from reimplementing the wheel more than we need to. Work began towards the end of 2019, building on top of Ruma’s excellent Matrix rust crates, and poljar has been working away solidly at it ever since. We teased matrix-rust-sdk in last year’s update, but as of this year it is properly coming of age and we’ve started using it in earnest - beginning by swapping out Element Android’s encryption implementation for matrix-rust-sdk-crypto (the E2EE cryptography crate provided by the SDK).
If you’re not familiar with Rust, the main benefits we get here are a heavy emphasis on safety and security without compromising performance; while providing a single codebase which can be used equally from iOS, native Desktop apps such as Fractal, Android (with native bindings) and even Web (via WASM, in future). While technically this results in a “non-native” SDK relative to matrix-js-sdk, matrix-ios-sdk and matrix-android-sdk - in practice, it’s become so common to depend on native-code shared libraries (outside the web, at least) that it’s not really a problem.
Initial results look wildly promising here: “Element R” (formerly known as Corroded Element - the codename for the Rust-enhanced version of Element Android) builds are now out there, and out-perform the kotlin E2EE implementation by roughly 10x, thanks to using native code and Rust’s improved parallelisation.
Our next step is to start using it on iOS, and we’ll be experimenting with a next-generation of Element iOS shortly in the new year with the SDK provided exclusively by matrix-rust-sdk. Element will also be funding more people to work fulltime on matrix-rust-sdk itself, and to see what the developer experience is like when you use it seriously on the Web - watch this space!
Bridges, Bots, Widgets and Integration Managers
Elsewhere in Matrix, the Bridge Crew has busy polishing bridges like crazy - working away on encrypted application services (MSC3202), massively improving the IRC bridge (particularly in the fallout of the great Freenode->Libera migration), stabilising and extending matrix-bifrost (our XMPP-and-more bridge), getting libpurple bridging working properly in bifrost, getting matrix-appservice-slack and matrix-appservice-discord stable enough to be hosted by EMS, experimenting with matrix-bot-sdk as an alternative bridging API, and even looking at adding matrix-rust-sdk-crypto into matrix-bot-sdk as an elegant way to power robust encrypted bridges (thus replacing Panatalaimon for that use case).
There’s also a new kid in town: matrix-hookshot (formerly known as matrix-github) is a new all-singing-all-dancing general purpose integration built on matrix-bot-sdk, coming soon to an integration manager near you, which can bridge through to GitHub, GitLab, JIRA and freeform webhooks! Check it out a few weeks ago on Matrix Live. matrix-hookshot is primarily Node, but is also getting in on the Rust action with some functions being implemented in native code.
Meanwhile, change is afoot for integration managers, which have been screaming out for an overhaul for years. There was a cheeky hint in last week’s Matrix Live where Dimension did an unexpected cameo looking particularly swish… All shall be revealed next year!
Dendrite, Low bandwidth Matrix and Peer-to-Peer Matrix
Dendrite is our next-generation homeserver implementation written in Go, and having shipped the first beta in Oct 2020, we’ve cut another 11 releases over the course of this year - adding in features such as E2EE key backups, cross-signing, support for room versions 7, 8 and 9 (knocking and restricted join rules), massive state resolution performance improvements, an entirely new state storage implementation that uses ~15x less disk space, sync filtering, experimental support for peeking-over-federation (MSC2444) - not to mention huge numbers of bug fixes. Even more excitingly, we’re in the process of ditching Kafka in favour of native-Go message queuing in the form of NATS!
However, it’s been a bit of a weird year as the team has been repeatedly pulled onto other projects due to competing priorities - and there’s still a bunch of stuff left which is keeping us in beta. Some of this is plain old missing features (search, push rules/notifications, room upgrades, presence etc) - but we’ve also run up against some problems over the last few months while implementing new room versions and similar thanks to the sheer number of different microservices which Dendrite is made out of. In retrospect, it feels like Dendrite has ended up too granular, and when hacking on it you get slowed down badly by all the boilerplate required to glue the various services together. Therefore, we’ve just started to merge some of the services together - still preserving horizontal scaling of course, but refactoring the architecture a bit while we’re still in beta to help speed up development again. So far things are looking promising! We’re also really looking forwards to s7evinK joining the team to work on Dendrite fulltime in the coming weeks :)
Talking of competing priorities, there have been three other big missions going on at the same time as Dendrite dev: firstly - formalising Low Bandwidth Matrix. LB Matrix is super important for maximising battery life on mobile, as well as (obviously) supporting worse network conditions - and it’s effectively a prerequisite for P2P Matrix. We did a bunch of experiments around it back in 2019, but earlier in the year we needed it for real and MSC3079 was the result. The low bandwidth dialect which we’ve proposed in the MSC is designed for use on the real Internet using standard IETF protocols (CoAP + DTLS + CBOR) and so isn’t quite as exotic as the 2019 version, but still gives a ~10-20x bandwidth improvement over normal HTTP+JSON based Matrix. It hasn’t made it to Element yet, but if you’re interested go check out the blog post!
Secondly, we’ve been sidetracked by the entirety of P2P Matrix. This is our long-term mission to let Matrix run peer-to-peer without the need for any servers (or indeed Internet connectivity, thanks to Bluetooth Low Energy) by embedding servers such as Dendrite into clients such as Element and so let each Matrix Client have its own personal local homeserver. We’ve made massive progress over the course of the year on P2P - the biggest breakthroughs being Pinecone as an entirely new P2P overlay network, with the novel SNEK (sequentially networked edwards key) routing topology. (The animation below shows a P2P network arranging itself into a SNEK!)
You can read all about it in the blog post, but suffice it to say that Pinecone outperformed all the other P2P overlay networks in meshnet-lab’s Mobility2 test:
You can play with P2P Matrix today on iOS and Android (head over to #p2p:matrix.org for builds), but there is some major work still to be done:
We need to bridge to today’s Matrix network. Right now, having a weird experimental test network for P2P means that in practice nobody actually uses it other than for demos - whereas if you could actually talk to everything else in Matrix, it’d be way more compelling and interesting to use and dogfood. We’re currently thinking about how best to do this!
We need to standardise the actual transport to be used over Pinecone. Currently it uses HTTPS over μTP (purely because empirically it handled packet loss and congestion well, and LB Matrix wasn’t ready at the time). We’re currently experimenting with switching to LB Matrix using our own CoAP implementation called PineCoAP (potentially using pCoCoA congestion control, given CoAP doesn’t provide any congestion control out of the box), but this is early days.
We still need to finalise store-and-forward: if your destination is offline, do you buffer your transactions in the network somehow, or do you use another Matrix node to buffer them?
Relatedly, we need to tweak federation so that if events get lost, federation for a room can recover more gracefully than it does today - for instance, by bundling redundant auth events on transactions, or by providing more recovery mechanisms.
We still need to spec and implement multihomed accounts, so that your identity on your phone is not divorced from your identity on your laptop.
…and obviously, we need a robust post-beta Dendrite to act as the local homeserver!
Right now focus is going back to Dendrite for a bit, but P2P work will resume again in the new year :)
Finally, the third big distraction from Dendrite has been… sync v3.
Sync v3
Sync v3 is shaping up to be the single most significant improvement to Matrix since we began.
Syncing data from the homeserver to the client is obviously fundamental to Matrix - and the current behaviour (sync v2) is far from perfect, as it’s designed around the assumption that your client wants to receive information for every room that it’s in. In the early days of Matrix, this was fine: a typical user might be in tens of conversations, and it’s useful to have them all available for offline access. Nowadays, however, it’s a disaster: users can easily accumulate hundreds or thousands of rooms - especially with rooms used to describe spaces or profiles and other structured realtime data. Moreover, the number of rooms you’re in typically increases linearly over time, unbounded, as nobody wants to archive their old conversations.
So, the idea of sync v3 is that you only sync the strict subset of data that your client actually cares about to display in its UI - effectively making both initial and incremental sync instant, incredibly low bandwidth, and completely independent of the number of rooms you’re in (just as filesystem performance should be independent of the number of directories or files present).
For instance, the full initial sync for @matthew:matrix.org in sync v2 is 417MB of JSON uncompressed - or ~100MB if gzipped, and takes about 5 minutes to calculate on matrix.org (during which it murders the sync worker responsible and hammers the database like crazy). By contrast, sync v3’s initial sync is 15KB uncompressed, or 5106 bytes compressed - and synced in 250 microseconds from a local sync-v3 server. Yes folks, that’s somewhere between a 30,000x to 1,200,000x improvement over sync v2, depending on how you count it.
Sync v3 gets this unbelievable performance by the client defining a sliding window into the server’s datasets, sized and ordered as needed for the client’s UI. This effectively performs real-time serverside pagination, so that as the client scrolls around or filters their roomlist or membership list, the client requests new views from the server. Meanwhile the server sends incremental updates to the client if they intersect with the sliding window. This may sound unwieldy, but in practice it works fine (although we’ll have some interesting challenges when we get around to encrypting state events, given serverside ordering and filtering will become distinctly harder). It also doesn’t design out offline access, as the client caches its view of the world so even if you do go offline you can still work with all the data that has sent to your client so far (and the client could even proactively paginate in other content, if it wanted to, similar to an email client synchronising for offline access).
Sync v3 exists today as a proxy called sync-v3 which sits between any existing homeserver and a sync-v3-capable Matrix client. It’s very early days, but Hydrogen has basic v3 support on a branch which we’ve been using to experiment with the API and flesh it out - and you can see a demo and intro talk in last week’s Matrix Live!
The API itself is still in flux, but those interested can see the initial spec design at https://github.com/matrix-org/sync-v3/blob/main/api.md and also an MSC is emerging at MSC3575. Next steps will be to finish hooking up to Hydrogen (including filtering the room list), finish the MSC, and then start thinking about implementing it in other clients and servers!
Fast Joins over Federation
While we’re on the subject of speeding up Matrix… it’s all very well being able to sync your client instantly, but the other big complaint everyone has about Matrix is how long it takes to join rooms - especially big ones. As most people will know, it can easily take 5-10 minutes to join a large room like Matrix HQ on a new homeserver - and given this is the first experience most users have of running their own homeserver, it can prove pretty disastrous and we are determined to fix it. It will become even more relevant when we implement peeking over federation, as the last thing you want is to have to wait 5 minutes to temporarily dip into some random federated room to see if you want to join it or not (or to sniff its room state for things like extensible profiles or MSC2313 reputation rooms).
So, to address this, we’re currently in the middle of experimenting with MSC2775 (Lazyloading over Federation) in Synapse. This MSC lets servers participate in a room before they’ve received the full room state by defining a subset of state which is mandatory for participation, and then letting the rest get added lazily. It’s quite a violent change as it means the assumption that room state is complete (to the best of the server’s knowledge) is no longer true - but given Matrix already has to handle incomplete room state, it’s not necessarily a showstopper.
Watch this space for how well it works in practice, but we’re hoping for a ~20x speed improvement in joining Matrix HQ.
Hydrogen
2021 has been a busy year for Hydrogen - our ultra-lightweight Matrix Client, which provides a small but perfectly formed progressive web app for us to experiment on! There have been no fewer than 56 releases over the course of the year, with loads of contributions from Bruno, Midhun (who joined first as a GSOCcer and then as a fulltime Element employee) and also Danila who interned at Element on Hydrogen over the summer.
People often ask why Hydrogen exists as well as Element Web - and the reason is because Element Web is (for now at least) very far from a progressive web app and is stuffed full of features, whereas Hydrogen is intended to be as lightweight and simple and efficient as possible while also targeting as wide a range of web browsers as possible (even Internet Explorer!). It also provides a simpler platform for experimenting with new approaches such as sync v3 or OIDC without getting entangled in the constant hive of activity around Element Web. Finally, it gives us a playground to experiment with embeddable chat clients thanks to Hydrogen’s strict MVVM component model.
In terms of features, 2021 has seen huge steps forwards as Hydrogen converges on feature parity with Element - proper mentions and replies; rich formatted linkified messages; reactions; redactions; memberlist; member info; webpush notifications; proper image, video & file uploads; SSO login; sync v3(!) and so much more. Can’t wait to see what 2022 will bring!
End-to-End Encryption
2021 saw the long-awaited creation of a dedicated cryptography team to focus exclusively on improving encryption in Matrix: previously encryption expertise was split across various different areas, meaning that it could prove hard to carve out time to tackle the bigger remaining encryption challenges.
So far the team has been busy digging deep into the few remaining causes of UISIs (undecryptable messages), including automated UISI reporting and tracing E2EE flows end-to-end (from client to server to server to client). There’s also been an initial wave of UX work - with much more to come next year as we overhaul cross-signing and device backups to make it way more user friendly.
Meanwhile, on the more foundational side of things, we’re continuing to define Decentralised MLS as a potential next-generation form of end-to-end encryption, building on the IETF’s MLS work - providing much better scalability for large chat rooms and potentially helping with some causes of encryption failures. Hubert (uhoreg) has been leading the charge here, with his latest thoughts emerging here alongside a brand new demo showing his DMLS simulator - which under the hood is actually sending real Matrix events over DMLS!
Otherwise, the team has had three big projects: adding matrix-rust-sdk-crypto into Element Android (which we already covered above), arranging a fresh security audit of Matrix’s end-to-end encryption (due to complete January 2022)… and, most excitingly: vodozemac.
Vodozemac (pronounced roughly vod-oz-eh-matz) is an entirely new implementation of our Olm and Megolm end-to-end encryption system, written from scratch in pure Rust, aiming to replace the original reference C/C++11 implementation in libolm. Originally written as an experiment for matrix-rust-sdk at the beginning of the year, in the last week it’s received a huge explosion of attention from poljar and dkasak to bring it up to production quality… for we decided that if we are doing a full E2EE audit for Matrix, we should target the new and future codebase rather than burn money on re-auditing the legacy libolm library (much as the original 2016 review of libolm happened when the library was fresh and new).
The motivation for vodozemac in general is to benefit from the intrinsic type and memory safety and fearless parallelism provided by Rust - and also maintain full type & memory safety throughout the matrix-rust-sdk stack, including encryption. Over the last year we’ve been taking more and more of a careful look at libolm, and despite our best efforts a few memory management bugs have crept in - which vodozemac should be immune to. Vodozemac will solve another embarrassing problem with libolm: that its default cryptography primitives are designed for correctness rather than performance or safety. By switching to Rust’s ed25519-dalek and rustCrypto AES primitives we should be in a much better position in terms of performance and safety.
Next up, we’ll be fully integrating vodozemac into matrix-rust-sdk, and figuring out how best to provide it as a libolm replacement in general.
Matrix Security
Alongside the new Cryptography team we’ve also established a new dedicated Security team for Matrix, led by dkasak. As well as fuzzing excursions into libolm and similar research, Denis has been handling all our security disclosure policy submissions, managing the Intigriti bug bounty programme, helping coordinate all our security releases, and coordinating the upcoming external independent security audit of vodozemac, matrix-rust-sdk, Element and Synapse. It’s a huge step forwards to be able to fund full-time infosec researchers to focus exclusively on Matrix, and this is just the beginning!
Trust and Safety
Another place where we’ve created a dedicated team this year is around Trust & Safety: building tools to fight spam and abuse on our own servers, while also empowering the wider network of users, moderators and admins to manage abuse as they see fit. This includes lots of work on Mjolnir, our primary moderation bot, but also defining MSCs such as MSC3215 (Aristotle: Moderation in all things) and MSC3531 (Letting moderators hide messages pending moderation) and internal tooling as we experiment with different approaches.
We’ll have more updates on this in the coming year as we release the tools we’ve been working on, but suffice it to say that the goal is to empower mainstream users in the wider Matrix network to apply their own rules as they see fit, directly from the comfort of their favourite Matrix client - without having to know what a Mjolnir is (or how to run one), and without having to be a moderation expert.
OpenID Connect
A new project brewing throughout 2021 has been the investigation into replacing the entirety of Matrix’s authentication APIs with industry standard OpenID Connect. Spearheaded by Quentin, this has proved to be a fascinating and challenging endeavour, but we’re starting to see some really interesting results. The problem we’re trying to solve here is:
As Matrix grows, we’re seeing more and more clients and services appearing which you might want to log into with your Matrix account. But do you really want to trust each app with your account password? And what if you only want to give it access to a small subset of your account?
Similarly, we’re seeing more and more login mechanisms used to access Matrix - it’s no longer just a matter just a username + password; many servers use single-sign-on (e.g. mozilla.org) or social login (fosdem.org, matrix.org), or layer on 2FA or MFA hardware tokens and similar to access their accounts via an SSO provider. We also see passwordless login on the horizon.
So, do we really want to mandate each new Matrix client to have to implement custom flows to handle this explosion of login/registration mechanisms? And is it even really the client’s problem in the first place? You’re securing access to your account on your chosen server, which isn’t really a client-specific thing at all.
The real turning point for the project however has been our recent experiences building out a new wave of single-use domain-specific clients (see below) for video conferencing, whiteboarding, metaverse-browsing etc… where by far the most painful bit of the project has been hooking up the UI for login, registration, guest access, incremental signup, password reset, email verification, CAPTCHA, SSO, etc. And that’s even when building on top of matrix-react-sdk, which theoretically has it all already thanks to Element Web!
Frankly, it has become blindingly obvious that it’s crazy for clients to reimplement this every time, and they should instead chuck the user over to a sign-on portal provided by their homeserver - just like Google and everyone else’s single-sign-on does. And rather than inventing our own homebrew way of doing that, we should just use the existing industry standard SSO best practices defined by OpenID Connect.
The main objections which have come up against this are: “what if my Matrix client doesn’t have a web browser, or what if I want to provide my own native login UI”, and “does this design out the idea of using a single password to access your account as well as your E2EE history”? In both instances, we have workarounds: in practice, there are so many Matrix clients around that we won’t be removing today’s legacy login/registration APIs any time soon (just like HTTP Basic Auth is still very much a thing on the web!). And in terms of “cryptographic login”, there are ways we could daisychain the auth required to unlock your E2EE storage to also authenticate you with your server - although this would be a major extension (much as cryptographic login is already today!)
The current status is that we’ve defined a set of initial MSCs (MSC2964, MSC2965, MSC2966 and MSC2967), and are implementing an initial Open ID Connect auth server (in Rust!) called matrix-authentication-service (better name suggestions welcome!) designed to sit alongside your homeserver, and we’re experimenting with hooking Hydrogen (and some of the new domain-specific clients) up to see how it feels. But if it goes as well as we think it might, folks should prepare for 2022 to be the year where Matrix’s authentication system finally gets fixed!
Native Matrix Video/VoIP Conferencing
One of the most anticipated features in Matrix over the years has been the prospect of native, decentralised, end-to-end encrypted video and voice conferencing. Today, voice and video conferencing in Matrix works by embedding Jitsi as a third party centralised service into your chatroom. This works fairly well - but Jitsi is an entirely separate service with lots of moving parts, and its own concept of users and access control (provided by XMPP!) and its megolm-based end-to-end-encryption doesn’t actually integrate with Matrix’s own Olm identities, verification or cross-signing. The fact that the conference is then logically centralised on whoever is hosting the Jitsi service also misses one Matrix’s main goals - that users should be able to hold a conversation without being dependent on any single service or provider. Plus it’s really confusing that Matrix has proper native 1:1 calls for DMs… but then switches to a totally different system in group chats.
So, this year we set out to fix it - and succeeded :D The solution hinges around MSC3401 - a spec proposal that describes how to extend native 1:1 calls to work for groups, while providing real flexibility on how to actually mix the calls together. At the simplest extreme, it defines how full mesh calls work (where every client simply calls every other client simultaneously) - but then also defines how you can mix calls together either using a single focus (conferencing server) or multiple foci run by different parties, where foci can either be Selective Forwarding Units (SFUs, like Jitsi) or Multipoint Conferencing Units (MCUs, like FreeSWITCH). The end result is to give us decentralised, cascading, end-to-end encrypted conferencing which even has direct compatibility with today’s 1:1 Matrix calling, letting you easily hook in bots and bridges which already support 1:1 Matrix calls!
Robert Long has been frantically hacking away at the initial implementation over the last few months, fleshing out full-mesh conferencing at first and getting it running in as many browsers as possible (including Mobile Safari and Chrome Android!). We were hoping to fully unveil the end result in time for Christmas, but in practice we hit some last minute snags (turns out Matthew forgot guest users can’t use TURN, who knew? so much for incremental login! 😰) which have pushed the launch to early next year. But hopefully in a few weeks, you’ll be able to start jumping on a native group call in Matrix!
Meanwhile, those interested can see all the gory details from our CommCon 2021 talk a few weeks ago, complete with a demo of the shape of things to come…
Next up, we’ll be working on building an MSC3401-compatible SFU so we can go beyond full mesh (which typically supports a maximum of ~7 callers). Our candidates right now are mediasoup, ion-sfu, janus and signal-calling-service - we’ll let you know how it goes! Also, if you’re interested in helping us build this out quicker, we are frantically searching for more WebRTC & VoIP gurus to join the team at Element working on this.
Applications Beyond Chat
Finally, 2021 was the year where we seriously started building out functionality on Matrix which goes far beyond plain old chat rooms.
Work began in the summer as a research project led by Ryan, formerly tech lead for Element Web - looking at ways to store hierarchical structured data into Matrix while preserving real-time semantics; effectively using Matrix as a collaborative decentralised object tree, providing CRDT (Conflict-free Replicated Data Types) to allow richer applications to be built on Matrix. This journey led him to create Patience as a test environment for building out these sort of clients, and meanwhile Timo (famous of The Board) joined the team to build out Full Screen Widgets in Element, providing a much better UI for beyond-chat experiments.
Meanwhile, Matthew Weidner and the Composable Systems Lab at CMU stunned us all by presenting a complete CRDT solution using Matrix named Collabs at Strange Loop 2021. This is really impressive stuff - the brave of heart can go and embed a Matrix-powered end-to-end-encrypted collaborative markdown editor straight into Element via Collabs by following the instructions here. In practice, Collabs works by serialising the CRDT updates as base64 blobs inside Matrix timeline events (hello Wave, is that you?), but we’re now investigating how you might reconcile this with maintaining a proper realtime object tree in Matrix.
It’s hard to overstate how powerful storing freeform tree CRDTs in Matrix would be. It could open up everything from decentralised encrypted collaborative document editing to collaborative whiteboarding and collaborative Figma-style (or Penpot- or Blender-style) design. You could even start storing an HTML DOM into a room, alongside its binary assets, giving you a multiplayer DOM to build on… and then imagine if you could store the syntax tree of the code operating on that DOM alongside it, in the same room. Before you know it, we will have created kind of some incredible Smalltalk / Croquet / Alan Kay nirvana where code is data and data is code and it’s all running live in some kind of decentralised encrypted multiplayer Metaverse :D
While we’ve been looking at storing object trees in Matrix, another obvious angle that has emerged is to use Matrix for encrypted decentralised file storage. MSC3089 is a proposal on how you might represent hierarchies of files in Matrix - where each room acts effectively as a directory of files, with spaces forming a directory structure (much as they do already in today’s Matrix), leveraging Matrix’s existing decentralised access control mechanisms to control who can access what. Combine such a file storage system with the collaborative editing capabilities mentioned above, and suddenly a really exciting proposition starts to emerge. We’re investigating this right now, and all will be revealed early next year…
Finally, and last but not least, Robert Long has been building on top of our shiny new Native Matrix Voice/Video Conferencing capabilities to use Matrix as the communication backbone for a truly open, equitable and interoperable vision of the Metaverse. The best way of describing it is to look at his awesome Third Room demo from the Open Metaverse Interoperability Group demo session in September:
Now, some folks will recall that since day one (in fact, since before day one) the hope for Matrix was that it might end up as the communications fabric of the Metaverse. We were about 4 years early when we first starting enthusing about this, and then still ahead of our time when we did the world’s first 3D Video calling over Matrix. However, it now feels like the world has finally caught up - and we’re in grave danger of being overtaken by a dystopia where the big tech companies balkanize the Metaverse into a series of closed proprietary user-exploiting walled gardens, much like today’s incumbent chat silos - but even worse.
This is our chance to fix it before it’s too late, and Element is funding a small but highly targeted team to focus exclusively on building out open interoperable Metaverse over Matrix - ensuring that collaboration in 3D (and 2D) spatial environments in future is decentralised, secure and standards-based. This obviously ties in directly with the rest of the Beyond Chat projects listed above: it’s early days, but it’s incredibly exciting to imagine where we could end up if this works!
Finally, a question which has kept coming up while working on Beyond Chat projects has been whether to implement this new functionality as Matrix widgets, bake them into existing Matrix clients, or build them as domain-specific dedicated Matrix clients. But perhaps we’re thinking about this all wrong: what if your Matrix client was just a browser for Matrix rooms? Some of these could be chatrooms. Some of these could be VoIP/Video conferences or Discord-style voice/video rooms. Some of these could be message boards or mailing lists. Some of these could be collaborative editors or whiteboards. Some of these could be 3D views into the metaverse. Some of these could be rendered via widgets; some could be rendered natively if the client knows how. And some of these could even be good old web pages(!!!).
Imagine if your Matrix client was effectively a genuine browser of arbitrary decentralised realtime content? If your view into a Matrix room was just that: a full window view into that room, be it textual or 2D or 3D - and your Matrix client was just a browser which added the necessary chrome and navigation to help you tab between rooms, login and logout, manage your encryption, track who’s in the room, track your notifications, etc.?
Meanwhile, if you’re in a web browser, you might hop into a lightweight single-page domain-specific webapp which happens to use Matrix for collaboration. Or if you’re in a Matrix client/browser, you could hop to the same matrix URL to get at the same functionality with all the supporting chrome and UI overlays sliding in as needed…
Perhaps the vision of Matrix as the missing communication layer of the open Web is more literal than we ever thought. Eitherway, it will be fascinating to see how Applications Beyond Chat evolves over the next year.
2022
Now, I dare you to cross-reference all of the above with last year’s predictions for 2021 to see how we did :D In practice, the only things from the list we haven’t got to are peeking-over-federation (although arguably fast joins are a key part of that), account portability, and restoring incremental sign-up (although our new clients have it!).
So, here go the predictions for 2022 (keeping it short, otherwise it’ll be 2023 before this blog post gets finished…):
Client polish and performance - our prime directive is to ensure that Matrix clients can be built with UX polish and quality which exceeds our centralised alternatives. In practice, this means:
Element must spark joy. Ensuring Element’s Information Architecture continues to be simplified and refined, and that nobody who knows how to use a computer hits a WTF moment when first using the app. Never again do we want to see someone on Twitter saying “I have no idea how to use Matrix”.
Instant launch. With Sync v3 and matrix-rust-sdk we hope to make Element launch instantly on all platforms - including initial sync.
Fast joins. We should never get bored while waiting to join a room or accept an invite.
Spaces. While Spaces are already a huge improvement in letting users organise and discover rooms, there’s still much more to be done:
Flair - Users who are members of a space should be able to announce it loud and proud with a Flair badge on their avatar, like we used to with the old pre-spaces Communities feature (MSC3219 being the potential proposal).
Synchronising access controls - You should be able to apply access controls based on whether a user is a member of a given group (so that if you invite them to #moderators:example.com, they automatically get made moderator in all the rooms in a given space). It looks likely that this will be implemented at last using joepie91’s MSC3216 proposal for Synchronized access control for Spaces (rather than Matthew’s original MSC2962 - an excellent example of the community steering the spec process :)
Bulk joins - It should be a one-button operation to join all the rooms in a space.
**Subspaces **- as more and more spaces emerge, the ability to navigate them as a hierarchy becomes more and more useful. We want to get to the point where we can turn off the Matrix.org public rooms list, and instead present a Space tree of all the good rooms we know about in Matrix… delegating over curation to the wider community; building a huge USENET-style hierarchy of where to go in Matrix. To do that, we need subspaces to sing!
Removing communities/groups, which will then be entirely superseded by spaces.
**Threads **go-live!
Location share go-live
Pinned messages, so the most important messages are always visible to everyone n the room
Starred messages, so you never lose a message ever again
Custom emoji, finally merging in all the custom emoji work from the community.
matrix-rust-sdk
Element iOS on rust-sdk
Element Android on rust-sdk-crypto
…and experiment to see how matrix-rust-sdk feels on Web? It’s a real shame that Daydream got archived…
Encryption
Vodozemac in matrix-rust-sdk, maybe even elsewhere.
**Updated E2EE Audit **spanning vodozemac, olm+megolm, matrix-rust-sdk… and a representative sample of a typical Element+Synapse deployment.
DMLS - getting to the point where we can experiment with it in real clients.
Encryption Agility - the ability to migrate encrypted history is going to become really important as we evolve our E2EE, whether that’s by adding in post-quantum algorithms, or moving from Megolm to MLS, or any other shifts. We will need to start thinking about it in 2022.
Next-generation MSCs
Aggregations - finalising the foundational MSCs for aggregations, at last
Extensible events - finalising the foundational MSCs for extensible events, at last
**Sync v3 **- finalising the MSC and implementing it in matrix-rust-sdk
Fast joins - getting them implemented in Synapse and Dendrite
Peeking over federation - getting them implemented in Synapse and Dendrite
Extensible profiles - who needs a facebook wall when you have a profile room on Matrix?
Open ID Connect - using OIDC as an alternative auth mechanism for new clients.
Gitter parity
Importing the Gitter archives into Matrix via MSC2716
Implementing excellent public static Matrix archives (replacing both view.matrix.org and gitter.im’s static views)
Transfiguring Gitter into a Gitter-themed Element
Dendrite
**Parity with Synapse **- and out of beta, with any luck!
P2P Matrix
Exposing the normal Matrix network via P2P!
Multihomed accounts
Store and forward (if only by relaying via other P2P Matrix nodes)
**Low bandwidth transports **- via PineCoAP or similar
Making federation robust in a highly disconnected network.
Hydrogen
**Daily Driver **- making sure that Hydrogen can be readily used as a daily driver Matrix client, even if it lacks full parity with Element.
**Embeddable Hydrogen **- making the most of Hydrogen as a tiny lightweight PWA to embed it into existing websites.
Bots and Bridges
**Landing End-to-Bridge-Encryption **for all existing matrix-appservice-bridge based bridges
All the integrations!
First-class UI for configuring integrations!
Trust & Safety
Empower users to manage abuse within their communities.
Something we didn’t mention in 2021 is the increasing interest in building border gateways and hardware cross domain gateways to safely link different Matrix federations together. We expect to see a lot of activity in this space in 2022, and there should be some new MSCs too :)
Beyond Chat
**Metaverse on Matrix **- building out the dream as per above!
**Collaborative editing **- extending Matrix to store trees of events, and collaborate on them in realtime - starting with a collaborative editor!
File storage in Matrix - building out real-life file storage on top of Matrix.
So, there you have it. If you’ve got this far… it’s incredible; you’re amazing: thank you for reading! The sheer length of this update shows just how much Matrix has grown in 2021 relative to previous years; it’s frankly terrifying to imagine how long the equivalent post will be next year. We may have to change the format a little :)
And that’s a wrap for 2021: we hope you stay safe and have an excellent end of the year. Huge thanks for flying Matrix and supporting the project - we literally wouldn’t be here without you.
Hey all, it's me again, not-anoa, with your spec update. I don't have a graph for you this week, but I do have curated content which hopefully holds you over until next week 🙂
This week we've seen a few new MSCs get opened up:
As demonstrated, a few of them are follow-on work from aggregations. A lot of the work is an effort to get MSC2675 - Serverside aggregations of message relationships through FCP - it's changed quite a bit in the last week, so if you reviewed it before then please give it a quick read!
Thib has caught me quite close to the deadline on TWIM, so while I don't have a random MSC for you this week I do recommend some light reading around aggregations and extensible events - these are both major features for Matrix and help shape the future for other, even more exciting, features.
The goal of this MSC is to restrict who can access files in your content repository, which is used to share files, images, voice messages and more on Matrix. Currently this is secured by using random identifiers, but this MSC wants to add a few more restrictions: A signed-in user can still access everything, but if you don't provide an access token, when downloading media, you either can access no media, only media that was sent from that server, only media which the server already downloaded or everything. The level of access can be set by the server admin.
I think this would be really nice to have, but it has various challenges when implementing it in clients and when it needs to work across federation. It could limit how much random users can fill up your disk though, which is especially important for small server admins!
Happy holidays! This week we released Synapse 1.49.0, our last planned release of the year. This is the last release that supports Python 3.6 or PostgreSQL 9.6; if you've not upgraded, now is the time!
Most notably, this release includes stable support for MSC2918: Refresh tokens. This is a more secure alternative to long-lived access tokens, and we'd encourage clients to implement support for refresh tokens as described in the MSC.
We also released Sygnal 0.11, which includes loads of bugfixes. For the Element-managed Sygnal instances, this release has reduced our daily Sentry error rate by over 99%, dramatically improving the signal-to-noise ratio of our monitoring.
Sydent is the reference Matrix Identity server. It provides a lookup service, so that you can find a Matrix user via their email address or phone number (if they have chosen to share it).
I've just published the third and final post on Sydent's type annotations (part 1, part 2). This one is more reflective and tries to quantify our efforts: how well have we done?
We are planning to have a small release next week, that fixes a few issues with the 0.9.0 release. It would be lovely if some of you could test one of our nightlies or could check if the translations for a language you speak are up to date in our weblate.
Some of the fixes this week include another crash fix for handling matrix links from your browser, notification bubbles that can show values over 9000(!), better preview images for sticker and emote packs created in Nheko, allowing you to click links in replies, a few layout and click area fixes. Nheko now also keeps track of your latest reactions and gives you easy access to them in the hover menu.
Nheko now also finally supports pinned messages! Most of you probably don't know, but that feature has pretty much always been in the matrix spec, but very few clients expose it. Today Nheko joins that rank! It's part of our goal to provide better support for building communities. Topics can be quite limiting, because they can only contain plain text. Pinned messages allow for much more creative freedom! They can also be encrypted, while state events currently never are, but the key for that isn't reshared, so currently experience in encrypted rooms is a tradeoff. Maybe we'll go for an encrypted description event in the future, but for now this seems to be a good solution to bridge the gap.
Let's hope the current master branch is good and we'll have a release with ALL THE FIXES next week! And thank you everyone, who already translated and reported issues! It took less than 10 hours to have 5 languages updated to 100%! Last year we didn't even have that many languages at 100%! You guys are AMAZING! <3
Development has kicked off for location sharing! Watch this space for more news in the new year.
Threads
List of threads in a room is now more accurate and viewing very long threads has improved with the integration with the homeserver APIs. (These APIs are not enabled on matrix.org yet.)
Threads on mobile platforms are catching up to Web, with many changes in review.
Iterated on the design for restricted history threads and search results across all platforms.
Design for thread previews in room list has been improved for mobile platforms.
Improved Android bottom sheet expandable and scrollable behaviour design.
We’ve created MSC3567 to fix some edge cases with API calls
Polls
Only one Android bug is outstanding!
Gathering feedback to incorporate into the next phase of development, join us for the community testing session on Monday at 17:00 in #element-community-testing:matrix.org
VoIP
Continuing with polishing & bug fixing for full mesh calling app. One remaining bug somewhere causing members to not connect properly. Finalising how much of registration & login we can/want to implement for the short term until it’s replaced by OAuth login.
Community testing sessions
Tested all three Release Candidates (RCs) at the same time for the first time! Did not find any new web issues when testing first time user experiences and basic interactions on Web. We found 6 new issues on Android and 9 on iOS.
Tried out the Information Architecture changes on web with the Delight team. Very exciting to see these changes available in Labs already!
Closed 33 out of 60 re-tested encryption issues on Web and prioritised a few to be considered for upcoming work.
We’ve been working on auto-generating code and documentation for events raised by our client analytics (code here, PR for documentation generation here). This allows us to publish a comprehensive list of everything our analytics capture, which is great both for end users and for people doing analysis.
In labs (you can enable labs features in settings on develop.element.io or on Nightly)
First milestone reached on Information Architecture! To try it out, enable “Threaded messaging”, “Use new room breadcrumbs” and “New spotlight search experience” in the Labs settings.
We’re actively collecting feedback on IA to review in the new year.
Fixed an issue around some voice messages not playing in bridged rooms.
Polls changes are in this week’s release candidate (RC) and will be available behind a Labs flag in the next release.
Analytics changes have been merged into the RC and opt-in will be available soon.
In development:
Spaces is coming closer to completion: we’re there on space creation, adding rooms to spaces, space management and more. Release coming in the new year!
Work on implementing new login flow continues, with more improvements incoming.
Improvements to the timeline performance (faster display, faster scroll) after updates to the way we store timeline events.
Analytics framework has been merged, opt-in request will be shown to users once more translations have landed. For now, you can enable it in the settings.
Find more about Cinny at https://cinny.in/
Join our channel at: #cinny:matrix.org
Github: https://github.com/ajbura/cinny
Twitter: https://twitter.com/@cinnyapp
I've been working on a matrix-based tool for highlighting and annotating websites. By building on top of matrix, we can effectively have a decentralized, federated and collaborative way to leave notes and highlights on pages. I wrote a brief introduction on my blog, as well as made a little bit of a simple demo video. Here's a copy-pasted list of planned and existing features:
Current: Create and send website annotations over Matrix.
Current: Store data in a decentralized and federated manner.
Current: Share highlights with other users, including those on other servers.
Current: Group annotations together and create multiple annotation groups
Planned: Use Matrix's End-to-End encryption to ensure the secure transmission and storage of highlight data.
Planned: Leverage the new m.thread MSC to allow users to comment on and discuss
highlights.
Planned: Use something like ArchiveBox to cache the current version of a website and prevent annotations from breaking.
Matrix highlight for probably the last time this week. Highlight comments and self-editing are implemented, though I'm not sure I'll stick with this particular model.
I've been teaching a class this semester using a tool built on the matrix-js-sdk and tentatively entitled populus-viewer. Populus-viewer uses Matrix as a backend for the social annotation of PDFs, with the goal of helping matrix become a platform for teaching and scholarly collaboration. If you're interested in learning more, or adopting populus-viewer in your teaching, come visit #opentower:matrix.org!
Populus-Viewer currently supports:
Annotation of PDFs with highlights and pin-drops
Matrix conversations based on annotations
Audio and video messages
Replies, reactions, and redactions
Markdown for rich text
LaTeX for mathematical notation
Typing notifications
Synchronized reading position across devices
SSO, with single-click links for embedding in an LMS like Canvas or Blackboard.
As the project develops, I'm hoping to continue to polish the reading experience, and to add support for other mime types (audio and video especially).
TWIM I got started on the chat part of matrix-streamchat, to provide a lightweight embeddable Matrix client to be used alongside streams in Owncast and PeerTube. It will use guest access, and lots more features to come like extensive custom emote support. For now refactoring a bunch of things first before adding more flashy things, but who knows, you might see me do it live on https://stream.pixie.town
Introducing vodozemac (https://github.com/matrix-org/vodozemac) - a rewrite of libolm in Rust by poljar and dkasak! The intention is for this to become the reference Olm implementation going forwards, and to get it audited asap (and benefit from all of Rust’s nice safety and parallelism features, and better crypto primitives!)
simplematrixbotlib is an easy to use bot library for the Matrix ecosystem written in Python and based on matrix-nio. Version 2.5.0 adds improvements to the config feature.
Feature Changes:
Add allow/block lists: This allows bot developers to specify allow/block lists of users who have permission to interact with the bot using regex.
Permissions can checked with Match.is_from_allowed_user(), which lets the bot developer choose which responses are restricted.
The allow/block lists can by modified at runtime via the Config.add_allowlist(), Config.remove_allowlist(), Config.add_blocklist(), and Config.remove_blocklist() methods.
A thank you to HarHarLinks for their contributions to version 2.5.0!
Dimension, an integration manager alternative for Element, has received a bunch of updates over the last couple weeks:
Added (early) support for matrix-hookshot's GitHub, Jira, and Webhooks bridging.
Most of a redesign complete to make it feel more like an Element UI rather than something special and third party.
If you're interested in helping out in getting the redesign finished, please check out https://github.com/turt2live/matrix-dimension/issues/458 which has reference mockups and linked issues. The major parts are the "complex bots" (Travis CI, RSS, etc) and the sticker integration. Unfortunately, I don't have enough free time to work on it myself in the near term, but will get back to it eventually 🙂
And now, a complementary screenshot of the Good™ parts:
The homeserver should be in the form of https://domain.tld
The username should be the username, not the user id. (krazykirby99999, not @krazykirby99999:matrix.org)
The room_id should be the internal room id of the room, not the published address. (!QQpfJfZvqxbCfeDgCj:matrix.org, not #thisweekinmatrix:matrix.org) This can be found under Room Options > Advanced > Room Information in the Element Client.
Other
If the room_id is not specified, the bot will send the message to all joined rooms.
If the message is not specified, it will default to Commit:.
The bot will join all invited rooms upon the start of an action.
Contributions are welcome - https://github.com/KrazyKirby99999/matrix-commit
matrix-imposter-bot - A bot that uses your account to repeat other people's messages. This gives relay-bot capabilities to puppet-only bridges.
I made this project a while ago to hack in a relay mode to the mautrix Facebook bridge. But as of this week, that bridge supports relaying natively! 🎉 Thus, my main motivation for maintaining imposter-bot is obsolete, and the project will be on indefinite hiatus.
With that said, it can (mostly) still be used to add relay support to any bridges that don't yet support a relay mode themselves--but native relay support is always better!
Thanks to everyone who's shown interest in the project, and to tulir for making such great bridges!
A project has started to re-implement the venerable mIRCstats for Matrix! It's in very early stages, right now it only does "getting a data-frame of events for a list of rooms" and has no actual visualisations baked in yet. However, we're moving quickly, and I hope to have some initial easy-to-use viz in place over the Christmas break.
The project is written in R (because I am an R user, and its good for data and viz work :P) and you can find it here. If you're new to R and want to give it a go, check out the extremely brief howto I just wrote here. I look forward to all the ways you will tell me it's broken!
Board games are great. And Matrix and its widget api turned out to be an excellent environment to create collaborative board games. With some really impressive conditions:
I don't have to maintain a server with a database.
I don't have to create a custom account systems user need to register. They play with their matrix account, which also makes accessibility great. Someone invites you in a room with the game and you can play!
I just need to host one static file and ppl will be able to play as long as that static site exists.
This project tries to be two things. A tech demo and inspiration to what is possible with widgets (Especially, with the changes on how widgets can be displayed in element (Check the "matrix live" Demos! 😊) )
Second it should serve as source and resource. For ideas and solutions on how no trust games can be executed without server (third party) side logic. And, for the ones interested, also as a resource on how widgets are implemented.
Last but not least the game Saint Petersburg is really fun. It takes a couple of minutes to grasp the rules but it is one of those games where there are so many things that can be considered with simple rules that it becomes more and more exciting with each round. So I really invite you to check out the rules and give it a try. Its best to start in the Git Repo or join the this room: #st-petersburg-auth:matrix.org
To put it simple, the widget works like this: The game state is stored in the room state and is updated through the widget directly.
This of course raises questions: How is it still possible to prohibit users from cheating and manually changing parameters like, how much money they own. Everyone (who has the permission) is always able to send whatever state events they want?
How is it possible to draw random cards if there is no third party involved. Could I not just send a state event with the cards that I hope are going to be drawn and are beneficial for me. Can we make card drawing deterministic? Not really since then everyone know what is going to happen. Which kind of breaks the game...
I would be super happy, if someone is interested and wants to find answer to the questions above by checking out the README.
There is an article (exclusive to paid subscribers) in the German tech news/magazine website heise.de about
"Running your own messaging service using the matrix server"
https://www.heise.de/ratgeber/Eigener-Chatserver-Mit-dem-Matrix-Server-einen-Messaging-Dienst-betreiben-6289020.html
This is the third in a series of three posts which discuss recent work to improve type annotations in Sydent, the reference Matrix Identity server. Last time we discussed the mechanics of how we added type coverage. Now I want to reflect on how well we did. What information and guarantees did we gain from mypy? How could we track our progress and measure the effect of our work? And lastly, what other tools are out there apart from mypy?
The best parts of --strict
While the primary goal was to improve Sydent's coverage and robustness, to some extent this was an experiment too. How much could we get out of typing and static analysis, if we really invested in thorough annotations? Sydent is a small project that would make for a good testbed! I decided my goal would be to get Sydent passing mypy under --strict mode. This is a command line option which turns on a number of extra checks (though not everything); it feels similar to passing -Wall -Wextra -Werror to gcc. It's a little extreme, but Sydent is a small project and this would be a good chance to see how hard it would be. In my view, the most useful options implied by strict mode were as follows.
By default, mypy will only analyze the implementation of functions that are annotated. On the one hand, without annotations for inputs and the return type, it's going to be hard for mypy to thoroughly check the soundness of your function. On the other, it can still do good work with the type information it has from other sources. Mypy can
infer the type of literals, e.g. deducing x: str from x = "hello";
lookup the return types of standard library calls, via typeshed; and similarly
lookup the return types of any annotated functions in your code or dependencies.
The information is already available for free: we may as well try to use it to spot problems.
This flag forces you to fully annotate every function. There are less extreme versions available, e.g. --disallow-incomplete-defs; but I think this is a good option to ensure full coverage of your module. It means you can rely on mypy's error output as a to-do list.
One downside to this: sometimes I felt like I was writing obvious boilerplate annotations, e.g.
def __str__(self) -> str:
...
There was one particular example of this that crops up a lot. Mypy has a special exception for a class's __init__ and __init_subclass__ methods. If a return type annotation is missing, it will assume these functions return None instead of Any. (See here for its implementation.) This is normally compatible with --disallow-untyped-defs and --disallow-incomplete-defs, with one exception. If your __init__ function takes no arguments other than self, mypy won't consider it annotated, and you'll need to write -> None explicitly.
It's also worth mentioning --disallow-untyped-calls, which will cause an error if an annotated function calls an unannotated function. Again, it helps to ensure that mypy has a complete picture of the types in your function's implementation. It also helps to highlight dependencies—if you see errors from this, it might be more practical to annotate the functions and modules it's calling first.
If I've written a function and annotated it to return an int, mypy will rightly complain if its implementation actually goes on to return a str.
def foo() -> int:
# error: Incompatible return value type (got "str", expected "int")
return "hello"
If mypy isn't sure what type I'm returning though, i.e. if I'm returning an expression of type Any, then by default mypy will trust that we've done the right thing.
Enabling --warn-return-any will disable this behaviour; to make this error pass we'll have to prove to mypy that i_promise_this_is_an_int() really is an int. Sometimes that will be the case, and an extra annotation will provide the necessary proof. At other times (like in this example), investigation will prove that there really is a bug!
This is a bit like a limited form of gcc's -Wtautological-compare. Mypy will report and reject equality tests between incompatible types. If mypy can spot that an equality is always False, there's a good chance of there being a bug in your program, or else an incorrect annotation.
I'm not sure how general this check is, since users can define their own types with their own rules for equality by overriding eq. Perhaps it only applies to built-in types?
Quantifying coverage
It was important to have some way to numerically evaluate our efforts to improve type coverage. It's a fairly abstract piece of work: there's nothing user-visible about it, unless we happen to discover a bug and fix it.
The most obvious metric is the number of total errors reported by mypy. Before the recent sprint, we had roughly 600 errors total.
dmr on titan in sydent on HEAD (3dde3ad) via 🐍 v3.9.7 (env)
2021-11-08 12:35:37 ✔ $ mypy --strict sydent
Found 635 errors in 59 files (checked 78 source files)
This is a decent way to measure your progress when working on a particular module or package, but it's not perfect, because the errors aren't independent. Fixing one could fix another ten or reveal another twenty—the numeric value can be erratic.
Reports
I found mypy's various reports to be a better approach here. There were three reports I found particularly useful.
--html-report
This produces a main index page showing the "imprecision" of each module. At the bottom of the table is a total imprecision value across the entire project.
The precision for each module is broken down line-by-line and colour-coded accordingly, which is useful for getting an intuition for what makes a line imprecise. More on that shortly.
--txt-report
This reproduces the index page from the html report as a plain text file. It's slightly easier to parse—that's how I got the data for the precision line graphs in part one of this series. That was a quick and dirty hack, though; a proper analysis of precision probably ought to read from the json or xml output formats. Here's a truncated sample:
+-----------------------------------+-------------------+----------+
| Module | Imprecision | Lines |
+-----------------------------------+-------------------+----------+
| sydent | 0.00% imprecise | 1 LOC |
| sydent.config | 0.00% imprecise | 266 LOC |
| sydent.config._base | 0.00% imprecise | 31 LOC |
| sydent.config.crypto | 15.94% imprecise | 69 LOC |
| ... | ... | ... |
| sydent.validators | 0.00% imprecise | 61 LOC |
| sydent.validators.common | 7.35% imprecise | 68 LOC |
| sydent.validators.emailvalidator | 1.30% imprecise | 154 LOC |
| sydent.validators.msisdnvalidator | 1.34% imprecise | 149 LOC |
+-----------------------------------+-------------------+----------+
| Total | 5.95% imprecise | 9707 LOC |
+-----------------------------------+-------------------+----------+
--any-exprs-report
Selecting this option generate two reports: any-exprs.txt and types-of-anys.txt. The latter is interesting to understand where the Anys come from, but the former is more useful for quantifying the progress of typing. Another sample:
The breakdown in types-of-anys.txt has more gory detail. I found the "Unimported" column particularly interesting: it lets us see how exposed we are to a lack of typing in our dependencies.
the proportion of "imprecise" lines across the project; I also used the complement, precision = 100% - imprecision, and
the proportion of expressions whose type is not Any.
These are plotted in the graph at the top of this writeup. I could see that precision and the proportion of typed expressions were correlated, but I didn't understand how they differed. I couldn't see an explanation in the mypy docs, so I went digging into the mypy source code. My understanding is as follows.
On balance, my preferred metric is the line-level (im)precision percentage. There wasn't much difference between the two in my experience, but the colour-coded visualisation in the HTML report is a neat feature to have. Maybe in the future there could be a version of the HTML report that colour-codes each expression?
It's worth highlighting the typeshed project, which maintains stubs for the standard library, plus popular third-party libraries. I submitted a PR to add a single type hint—it was a very pleasant experience! Microsoft has an incubator of sorts for stubs too.
Other typecheckers
After the sprint to improve coverage, I spent a short amount of time trying the alternative type checkers out there. Mypy isn't the only typechecker out there—other companies have built and open-sourced their own tools, with different strengths, weaknesses and goals. This is by no means an authoritative, exhaustive survey—just my quick notes.
Didn't seem to recognise getLogger as being imported from logging. Not sure what happened there—maybe something wrong with its bundled version of typeshed?
In a few places, Sydent uses urllib.parse.quote but only imports urllib. We must be unintentionally relying on our dependencies to import urllib.parse somewhere! Mypy didn't complain about this; pyright did.
Seemed to give a better explanations of why complex types were incompatible. For example:
/home/dmr/workspace/sydent/sydent/replication/pusher.py
/home/dmr/workspace/sydent/sydent/replication/pusher.py:77:16 - error: Expression of type "DeferredList" cannot be assigned to return type "Deferred[List[Tuple[bool, None]]]"
TypeVar "_DeferredResultT@Deferred" is contravariant
TypeVar "_T@list" is invariant
Tuple entry 2 is incorrect type
Type "None" cannot be assigned to type "_DeferredResultT@_DeferredListResultItemT" (reportGeneralTypeIssues)
/home/dmr/workspace/sydent/sydent/sms/openmarket.py
/home/dmr/workspace/sydent/sydent/sms/openmarket.py:93:13 - error: Argument of type "dict[_KT@dict, list[bytes]]" cannot be assigned to parameter "rawHeaders" of type "Mapping[AnyStr@__init__, Sequence[AnyStr@__init__]] | None" in function "__init__"
Type "dict[_KT@dict, list[bytes]]" cannot be assigned to type "Mapping[AnyStr@__init__, Sequence[AnyStr@__init__]] | None"
TypeVar "_KT@Mapping" is invariant
Type "_KT@dict" is incompatible with constrained type variable "AnyStr"
Type cannot be assigned to type "None" (reportGeneralTypeIssues)
This would have been really helpful when interpreting mypy's error reports; I'd love to see something like it in mypy.
Here's another example where I tried running against a Synapse file.
/home/dmr/workspace/synapse/synapse/storage/databases/main/cache.py
/home/dmr/workspace/synapse/synapse/storage/databases/main/cache.py:103:53 - error: Expression of type "list[tuple[Unknown, Tuple[Unknown, ...]]]" cannot be assigned to declared type "List[Tuple[int, _CacheData]]"
TypeVar "_T@list" is invariant
Tuple entry 2 is incorrect type
Tuple size mismatch; expected 3 but received indeterminate number (reportGeneralTypeIssues)
This is really valuable information. It's worth considering Pyright as an option to get a second opinion!
It looks like Pyright's name for Any is Unknown. I think that does a better job of emphasising that Unknown won't be type checked. I'd certainly be more reluctant to type x: Unknown versus x: Any!
PyCharm has its own means to typechecking code as you write it. It's definitely caught bugs before, and having the instant feedback as you type is really nice! I have seen it struggle with zope.interface and some uses of Generics though.
Runtime uses of annotations
When annotations were first introduced, they were a generic means to associate Python objects with parts of a program. (It was only later that the community agreed that we really want to use them to annotate types). These annotations are available at runtime in the __annotations__ attribute. There's also a helper function in typing which will help resolve forward references.
Programs and libraries are free to use these annotations at runtime as they see fit. The most well-known examples are probably dataclasses, attrs with auto_attribs=True and Pydantic. I'd be interested to learn if anyone else is consuming annotations at runtime!
Summary
All in all, in a two-week sprint we were able to get Sydent's mypy coverage from a precision of 83% up to 94%. Our work would have spotted the bytes-versus-strings bug; we understand why the missing await wasn't detected. We fixedothersmallbugs too as part of the process. As well as fix bugs, I've hopefully made the source code clearer for future readers (but that one is hard to quantify).
There's room to spin out contributions upstream too. I submitted twoPRs to twisted upstream; have started to work on annotations for pynaclin myspare time; and submitted a quick fix to typeshed.
Looking forward, I think we'd get a quick gain from ensuring that our smaller libraries (signedjson, canonicaljson) are annotated. We'll be sticking with mypy for now—the mypy-zope plugin is crucial given our reliance on twisted. We're also working to improve Sygnal and Synapse—though not to the extreme standard of --strict across everything.
I'd say the biggest outstanding hole is our processing of JSON objects. There's too much Dict[str, Any] flying around. The ideal for me would be to define dataclass or attr.s class C, and be able to deserialise a JSON object to C, including automatic (deep) type checking. Pydantic sounds really close to what we want, but I'm told it will by default gladly interpret the json string "42" as the Python integer 42, which isn't what we'd like. More investigation needed there. There are other avenues to explore too, like jsonschema-typed, typedload or attrs-strict.
To end, I'd like to add a few personal thoughts. Having types available in the source code is definitely A Good Thing. But there is a part of me that wonders if it might have been worth writing our projects in a language which incorporates types from day one. There are always trade-offs, of course: runtime performance, build times, iteration speed, ease of onboarding new contributors, ease of deployment, availability of libraries, ability to shoot yourself in the foot... the list goes on.
On a more upbeat note, adding typing is a great way to get familiar with new source code. It involves a mixture of reading, cross-referencing, deduction, analysis, all across a wide variety of files. It'd be a lot easier to type as you write from the get-go, but typing after the fact is still a worthy use of time and effort.
Many thanks for reading! If you've got any corrections, comments or queries, I'm available on Matrix at @dmrobertson:matrix.org.
There is currently a lot of buzz and uncertainty around a number of vulnerabilities discovered in the log4j library in the Java ecosystem. These vulnerabilities are collectively known as "Log4Shell" and currently encompass CVE-2021-44228 and CVE-2021-45046.
First and foremost, there are to our knowledge no Matrix homeservers written in Java. Synapse, the canonical implementation developed by the Matrix Foundation and the implementation that is backing matrix.org, is written in Python and thus unaffected. P2P Matrix relies on Dendrite, our next-gen homeserver which is written in Go and is unaffected. Conduit, a community homeserver, is written in Rust and also unaffected. Supporting components like Sygnal and Sydent are written in Python and unaffected.
There are two components that are commonly used in the Matrix ecosystem that do rely on Java. These are Jitsi, specifically the Jitsi Videobridge for VoIP, and signald used by the Signal bridge. Both components pull in log4j as part of their (transitive) dependencies. We're not aware of other bridges that are dependent on Java-based components.
For both of these projects updates have been published that integrate log4j 2.15.0 covering the initial CVE and we're currently waiting for additional updates to be published that integrate log4j 2.16.0 to cover the second. In the meantime, we've put all mitigations we are aware of in place on our systems and we strongly recommend everyone do the same.
For what mitigations to put in place, we recommend following the recommendations provided by LunaSec. They also provide a lot of background information on the vulnerabilities and how to audit for them.
Please keep an eye out for releases from the Jitsi and signald projects and follow their upgrade instructions to update your own deployments as soon as possible.
Synapse 1.49.0 is the last version of Synapse to officially support Python 3.6 and PostgreSQL 9.6. This follows our platform dependency deprecation policy.
As a consequence of this, Synapse 1.49.0 is the last version of Synapse to support Ubuntu 18.04 LTS (Bionic Beaver), as it ships with Python 3.6.
On the topic of supported Ubuntu releases, please note that Ubuntu 21.04 (Hirsute Hippo) reaches its own end of life on January 20, 2022. Past this date we will stop producing new packages for Ubuntu 21.04.
Improved documentation
Up until now, a lot of very useful information was stored on the Synapse repo's wiki, which wasn't well advertised nor well reviewed.
With this release, we have migrated most of this information to Synapse's documentation website, so all the information you need to set up, maintain and troubleshoot a Synapse instance lives at the same place. Included in these new pages are the server admin FAQ and a guide to Synapse's Grafana dashboard.
The media repository documentation has also been updated with a lot of details about how Synapse stores media files.
Refresh tokens
When a Matrix client needs to authenticate a request to a homeserver, it uses what is called an access token. Sometimes server administrators might not want a user's access token to live forever (e.g. for security reasons). To address this concern, MSC2918 introduces the concept of refresh tokens to Matrix.
Initial support for refresh tokens in Synapse was introduced in version 1.38.0. Synapse 1.49.0 finalises and stabilises this implementation, allowing any client that supports this feature to use it as it is currently described in the related MSC.
Everything else
This release introduces the last changes needed to Synapse for basic threading support. It also introduces support for MSC3030, which allows clients to jump to a specific date in a room's history (expect a sneak peek of this in the next episode of Matrix Live!).
Another interesting point is the addition of a couple of admin APIs for federation. More specifically, they allow you to visualise all of the other homeservers your Synapse instance has been interacting with, as well as how successful the last attempts at communicating with them have been.
Please see the Synapse release notes for a complete list of changes in this release.
Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including Dirk Klimpel, Maximilian Bosch and Tulir Asokan.
Till next year
This is the last release of Synapse of 2021! The Synapse team will take a break for the holidays, pushing the next release of Synapse (1.50.0) to January 11, 2022.
We'd like to thank everyone who has been using Synapse, contributing to it, and/or supporting us for the past year, and we hope to see you again in 2022! 🎆
Today we are releasing security updates to libolm, matrix-js-sdk, and several clients including Element Web / Desktop. Users are encouraged to upgrade as soon as possible. This resolves the pre-disclosure issued on December 3rd.
These releases mitigate a buffer overflow in olm_session_describe, a libolm debugging function used by matrix-js-sdk in its end-to-end encryption (E2EE) implementation. If you rely on matrix-js-sdk for E2EE, you are affected. This vulnerability has been assigned CVE-2021-44538.
Clients which do not use matrix-js-sdk for E2EE, like FluffyChat or Element Android / iOS, are not affected.
This issue has been present since the introduction of the olm_session_describe function in October 2019 (commits: libolm, matrix-js-sdk).
We do not believe it is practical to successfully exploit this issue. However, upgrading remains important as the overflow can be triggered remotely.
Separately from the above vulnerability, we noticed during an internal audit that the libolm bindings in matrix-js-sdk were not zeroing out certain arrays containing entropy for cryptographic operations. This causes the entropy to remain resident in memory longer than necessary. As a defense-in-depth measure, this release of libolm now proactively overwrites those arrays when it is safe to do so.
Lastly, we are also taking this opportunity to update the version of Electron bundled with Element Desktop, pulling in the latest backported security fixes there.
The buffer overflow was found and reported by GitHub user @brevilo in the course of developing jOlm, a library of Java bindings to libolm; thank you. If you believe you've discovered a security vulnerability in Matrix or its implementations, please see our Security Disclosure Policy for how to get in touch.
Last week the Matrix Scientists had turned this room into a portal to TWIM bot's ship tank so we could all fuel it with news about our work. The spec and hookshot news were reported late when the tank was already full, sending TWIM bot's ship into hyperspeed mode 🚀
We lost contact with the bot for a few days but managed to restore a connection with it. It appeared to have crashed on the green planet of Fuj'ehr. Its ship shattered in pieces at impact! 💥
The bot managed to find most of them, but some critical pieces of the engine were missing. We needed to find those pieces quickly before the mushy ground of Fuj'ehr swallows them forever! 😱
The Editor has asked the Matrix Scientists to reconfigure the room and bridge it to TWIM bot's navigation tools. Each news report highlighted the position of a piece of engine on its map. 🗺️
Matrix Live 🎙
This week my guests are the FluffyChat and MinesTRIX maintainers, following the v1 release of the simple and beautiful FluffyChat… and they want to work together!
Version numbers of each homeserver, updated daily using a GitHub Action, are now added onto my public homeserver list, in hopes that it will further aid those choosing a homeserver to use Matrix on.
Very handy to quickly know if a server is well maintained or not before making it your new home!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/unstable/proposals.
Extensible events are coming! With lots of potential usecases to be built on top of the concept (such as threading, polls, and any type of data one would like to through on top of existing room events), extensible events are finally getting some love. New MSCs are available above, which detail some of these usecases. Exciting times!
A small correction from last week's issue: the next aggregations MSC to be focused on (after MSC2675 (serverside aggregations) is MSC2677 (annotations and reactions), as it's a more pressing blocker for usecases such as threading and polls.
Otherwise, the Spec Core Team is continuing to wind down in preparation for the holidays ☃️
Quite a novel concept, and one that would enable many usecases, such as the ones described in the MSC itself.
Check it out if that's something that interests you!
This week we cut Synapse release candidate 1.49.0rc1 It includes a bunch of work to supporta plethora of MSCs (MSC 2675, MSC3030, MSC2918, MSC2946), a crop of bugfixes and improvements to our documentation which incorporate Synapse's old wiki. And as ever, there's a bunch of internal type hinting to keep the Synapse team's blood pressure at healthy levels.
The formal release of Synapse 1.49.0 is scheduled for the coming Tuesday. This will be the last Synapse release of 2021 as the Synapse team prepare for a break over the Christmas period. Releases will continue at the usual pace in the new year, with 1.50.0rc1 slated for 2022/01/04 and 1.50.0 for 2022/01/11.
Please note: Synapse 1.49 will be the last version to support Python 3.6, PostrgreSQL 9.6, and Ubuntu 18.04 LTS (Bionic): by our next release, these will have reached their upstream end-of-life. If you're reliant on any of these platforms, please ensure you have plans to upgrade.
In other news, we're preparing to release a new version of Sygnal with a series of fixes for common errors. This should make Sygnal administrators much happier by removing an awful lot of error spam from logs!
Sydent is the reference Matrix Identity server. It provides a lookup service, so that you can find a Matrix user via their email address or phone number (if they have chosen to share it).
This week has seen yet another set of updates to my Helm Charts, with element-web being bumped to 1.9.6 and matrix-synapse seeing fixes to non-standard port configurations and better support for modern ingressClass handling.
If you are using Nheko on a mobile device like the PinePhone, you should now be able to swipe between the room list and the spaces list. Since I don't use a PinePhone, feedback will be appreciated!
NeoChat version 21.12 is out! You may have noticed that this version number is roughly twenty times higher than the previous one. This means that NeoChat is now twenty times as good as the last version. Or it means that version numbers are utterly meaningless and we switched to a date-based version number system since NeoChat is now released together with many other plasma-mobile related apps. This also means that new versions will arrive monthly from now on. New features and fixes in this version include - but are not limited to:
Spell checking while writing a message
Improved markdown to html conversion when sending a message
Built-in theme switching
Various fixes to login, logout and account switching
Threads is making excellent progress; This week we held 2 internal testing sessions, both of which went swimmingly.
We’re continuing our hard-work on Notifications to fix those up as best we can.
Also, we started work on the Threads Filter. The filter will allow you to choose between all the threads in a room and threads you’ve actively participated in.
If you’re using the Labs version of Threads, let us know what you think so far!
Polls
Exciting news on Polls; All development on Polls MVP is nearly complete, and will soon be making their way to a production environment near you!
Polls will be available behind Labs flags at first. We're excited to see people using it and we’re looking forward to hearing any feedback/comments.
Community Testing
Closed 20 out of 36 issues in encryption and verification (E2EE) this week.
We are planning three testing sessions for next week:
Tuesday 17:00-18:00 UTC - first time user experience on iOS, Android and Web
Wednesday: 16:00-17:30 UTC - information architecture changes on Web (with Michael and Nique joining us from the Delight team)
Thursday 16:30-18:00 UTC - bug squash edition on encryption, can we get the issue count to an all new low?
If you've ever tried to find a message back in the past, you've experienced the burdensome task of having to scroll back manually for days, even months! With the jump to date headers, that will be a thing of the past 😌. Clicking any date separator in the room timeline, will give shortcuts to jump to last week, last month, jump to any date using the date picker, or even the beginning of the room to follow a room upgrade chain.
This is currently still in a draft pull request state but will give another update when it lands in Element Labs for everyone to use.
Element Android 1.3.9 has been released on the PlayStore and is available for the beta testers:
This version adds support for draft voice messages and a new design for URL previews.
Opt-in PostHog analytics will land soon, and will be included in the next release.
A new "Legals” screen has been added to Settings in order for users to see all legal info pages for Element, the user's homeserver and the user's identity server (if any).
The next release candidate will be prepared next Tuesday.
Commune is a communications suite built on top of matrix. Commune aims to bring together chat, discussions, email and other interactive apps into a single matrix client.
So here’s the thing ahq (dev) and I (product) have been working on: https://github.com/commune-org/commune
It’s a chat/forum hybrid. Still in pre-alpha, proof-of-concept stage.
TWIM I wrote a Matrix powered stream overlay for OBS, to integrate live chat in your favorite (selfhosted) streaming setups. Was a great little 2 evening project to develop while livestreaming it's development :)
You can find the code and instructions at https://git.pixie.town/f0x/matrix-streamchat
and a hosted instance at https://streamchat.pixie.town
jOlm has seen two releases since the previous update, v1.0.7 and v1.0.8. jOlm now supports (and requires at least) the latest libolm version 3.2.7. Please note that we deprecated a number of methods in favor of renamed siblings. The majority of the old ones will be removed in the upcoming jOlm 1.1 release, likely published soon after libolm's announced security release on Dec. 13th. Please follow suit and update your implementations accordingly.
Summary:
🧰 Maintenance and upstream update releases
⚠️ Renamed methods for improved coherence and due to upstream changes (old ones deprecated)
✅ Up to date with Olm 3.2.7 (new minimum requirement)
If you've got room for more video content, I also said more things about bridges. In this one, we do a live code session for a twilio bridge and watch it fly! https://www.youtube.com/watch?v=S5q3FLLvRn4
Wily has launched an iOS Matrix client to enable messaging in restricted- or poor networks. mText and Room events are transferred as DNS payload, thus bypassing most captive portals, while message headers are minimized to enable messaging in very low bandwidth/high latency networks.
Wily Messenger is in POC stage, missing i.e. encryption at the moment, among others. We are committed to develop it further and invite a Kotlin developer to join our journey. DM @hp:hq.wily.im
We've set up a new Element Space for the Element family of clients and projects, finally. Feel free to join it at #community:element.io and be sure to check out #community:matrix.org while you're there for everything Matrix related.
Join #ping-no-synapse:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
envs.net
138.5
2
construct.supercable.onl
165
3
weasy-is-my.name
317
4
grin.hu
403
5
dendrite.supercable.onl
531
6
spacedn.com
534.5
7
matrix.org
828
8
matrix.awesomesheep48.me
1084
9
dendrite.s3cr3t.me
1103.5
10
0x1a8510f2.space
1332
The Adventures of TWIM bot continued
The members of the Federation have been very active and helped TWIM bot to find all the pieces of its ship! TWIM bot assembled everything together and put the engine back in its place... but at the last moment, as it was ready to take off, another bright spot appeared on the map following madlittlemods late report!
How could this happen? All the pieces of the engine were already there! Unsure of what to do, the bot asked Earth for directions. Earth confirmed: it was worth going to that new bright spot on the map.
The bot welded back the plate of the engine casing, made sure nothing could get into the ship in its absence, and started heading to the mysterious spot. The signal of our communication tools with TWIM bot weakens as it enters in the thick forest of Fuj'ehr…
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
This is the second in a series of three posts which discuss recent work to improve type annotations in Sydent, the reference Matrix Identity server. Last time we discussed the motivation for doing this work in the first place: the why. Now I want to talk about the how. How did we add annotations to individual files, and across the project as whole? What common idioms did we learn on the way?
The process of improving coverage
From experience adding typing to Synapse, we decided to annotate one module at a time. We configured a list of files in pyproject.toml which we knew passed mypy's checks. We could run mypy in CI to check that new PRs didn't introduce type problems in modules already covered.
From there, the workflow was
Choose a new module to annotate. Add it to the list of files in mypy's configuration.
Run mypy. See how many errors you get.
Choose an error. Fix it. Re-run mypy.
Repeat until no errors remaining.
Submit for review.
There are two parts in there which involve a choice. Being honest, I made those choices unscientifically: I tried to choose the easy tasks to do first. My first target was actually the entire sydent.util subpackage. Probably a bit too large for a first bite! My thinking was that util sounded like something with few dependencies that would have impact across the whole source tree.
Within a given module, I'd try to fix easier errors first: partly for confidence, partly to build up momentum, and partly to get myself familiar with that piece of source code. For example, I'd often start by telling mypy that mylist = [] was actually was a List[str] (rather than the generic List[Any] which it would use otherwise).
Picking and choosing easy targets works fairly well, but sometimes that means you end up fixing an error that's really a symptom of an earlier one. Other times fixing one error, e.g. by giving a return type annotation to a function—would solve a series of other errors throughout the file. Watching the total number of errors mypy reports bob up and down was intriguing!
In retrospect, I think it would be smoother to generate some kind of dependency graph for the package. I'm imagining a DAG where whose vertices are modules, and there's an edge A -> B if A imports from B. The sinks of this DAG (i.e. modules which don't depend on any others in the package) are the ideal place to start: you can get something strictly typechecked there without having to annotate a long chain of dependencies across other files. Another strategy would be to see which modules were the least precise according to mypy's reports—but more on those next time.
You're at the mercy of your dependencies
I think this is my single biggest takeaway from the process of adding annotations to Sydent.
I'll admit the phrasing is melodramatic, but I think it rings true.
Improving coverage boils down to giving the typechecker more information about your program. The more information it has, the more it can check—and the more errors it can spot. (Hopefully this doesn't make typing come across like a pyramid scheme.) If your dependencies aren't typed, mypy can't validate you're correctly providing inputs and correctly consuming outputs. You might have a bigger impact on overall typing coverage by annotating a dependency (directly or via stubs). I have a hunch that bugs are more likely in code that uses an external dependency: we're much more familiar with the details of our own source code compared to that of a third party we trust.
It's worth looking to see if your dependencies have a newer version including type annotations. Failing that, they may have a stub package added to typeshed and published on PyPI. I saw example of both cases when choosing how to configure mypy for Sydent. If some of your dependencies are under your control, consider annotating them—you'll feel the benefits across multiple projects pretty quickly.
Annotations when working with twisted
Twisted is Sydent's biggest dependency, and I certainly felt at its mercy! In particular, it has a few quirks which make it trickier to annotate applications using it. Here's a summary of the work we had to do to get those annotations working.
Partially typed modules and stubs
Early into the process, mypy reported that calling the function twisted.python.log.err was an error. It did so because I was running mypy in --strict mode. We'll talk more about why I did so and what this means next time; for now, it's enough to know that calling a function that isn't fully annotated from within a function that is constitutes an error under strict mode. twisted is partially annotated: many key modules and functions have type annotations, but others don't. I was reluctant to give up on --strict. Instead, I decided to stub the err function myself.
A stub is a cut-down version of a python function, class or module which lives in a .pyi file. All implementation details are removed; only type annotations remain. Stubs are useful when you want to write annotations for code you don't control. The typeshed library is probably the best example: a collection of third party stubs for the standard library, plus some popular third party packages. Microsoft's python-type-stubs is another example. They'd also solve the problem I mentioned at the end of the first part: I could use a stub to teach mypy that IResponse.headers was a Headers object.
Writing a stub for log.err was straightforward, thanks mainly to Twisted's thorough documentation. I chose to write it by hand, rather than use stubgen. I'd heard of the latter, but was reluctant to use it for a few reasons.
Twisted is a big project with many big files. I didn't want to commit huge stub files for me and my colleagues to maintain.
The more our stubs cover, the larger the risk of a stub becoming out-of-date with twisted itself. Upstream twisted is the best place for these annotations.
We only need annotations for the bits of twisted that we're using.
And, being honest: I wanted the low-level experience of writing stubs, cross-referencing between the source and working how to best capture its type semantics.
I think this decision to write a targeted stub made sense at the time. After all, twisted.python.logging.err is just one simple function! But for the project as a whole, I regret not using stubgen to generate module-level stubs. The reason for this is that a .pyi stub file accounts for an entire module: no more and no less. This means that the stubs I was writing for parts of twisted only covered the functions and classes I'd stubbed. Any existing annotations in the twisted source code would be ignored, along with any types that mypy was able to infer for itself.
I think it would have been more efficient to use stubgen to generate stubs and patch them up, rather than writing them. That would have helped avoid a few cases I encountered where stubbing one function would cause additional typechecking failures (because mypy was no longer examining the twisted source for that file). It would also have meant that I could just faithfully stub Twisted as it is; in practice, I would sometimes hesitate to stub to avoid having to cover another module. sydent.http was the most painful part of the source tree for this: that's where we make the most use of Twisted.
This is a decorator which allows us to write code in the style of async/await without actually using that syntax. (Twisted predates asyncio and the async/await syntax, introduced in Python 3.4 and 3.5 respectively. It was originally released in 2002, back when Python had released version 2.2.) We only use it in one place in Sydent nowadays, but I've seen used across Synapse too. I mention it here because it was a bit fiddly to annotate. Here's its use in Sydent:
Here, request is a generator function because its body uses the yield keyword. Yielding allows the function to relinquish control back to twisted's reactor, only for its execution to be resumed asynchronously in the future. The Generator type takes three parameters:
a YieldType, Deferred[Any];
a SendType, Any; and
a ReturnType, IResponse.
Why have I opted to use Any here, when we've seen (and will see) that this limits mypy's ability to run checks? The answer is that we yield two different types within the function. Firstly a routing result:
res: IResponse
res = yield agent.request(method, uri, headers, bodyProducer)
In this example:
We yield a value y: Deferred[Any].
That will later be resolved by twisted to an x: Any value. Twisted will send that value to request, and the execution continues.
This repeats, until we eventually return an IResponse.
I could more correctly annotate the YieldType as Deferred[Union[_RoutingResult, IResponse]] so that the SendType was Union[_RoutingResult, IResponse]. But this would mean we end up having some kind of type check at each yield point: a cast, or a type: ignore, or a runtime isinstance check. It didn't feel like it was worth the boilerplate, especially since I could narrow e.g. res: Any to res: IResponse with minimal effort.
This problem doesn't arise with using the async/await syntax:
$ mypy example.py
example.py:6: note: Revealed type is "typing.Coroutine[Any, Any, builtins.int]"
example.py:7: note: Revealed type is "builtins.int*"
Behind the scenes, I think that x = await foo() is really using the same mechanism as the inlineCallbacks approach.
An async def function is really a generator function behind the scenes.
When we await foo(), we yield the expression foo()
Then the machinery running our coroutine c will call c.send(x) to resume execution, where x is the value produced by waiting for foo().
With the await form, mypy knows two things:
the value foo() which was yielded should be Awaitable[T], and
the value x send to the coroutine should come from awaiting foo(), and therefore be of type T.
Mypy can't assume or enforce these rules for the yield form, which can yield and send whatever it likes. There's no reason why the send type should be related to the yield type. Here's a toy example:
from typing import Any, Dict, Generator
def generator_function() -> Generator[int, str, Dict[str, Any]]:
y: int = 10
x: str = yield y
print("Coroutine was sent", x) # -> Coroutine was sent hello
return {"got": x, "done": True}
coroutine = generator_function()
y = next(coroutine)
try:
coroutine.send("hello")
except StopIteration as e:
return_value = e.value
print(return_value) # -> {'got': 'hello', 'done': True}
All in all, the handling of inlineCallbacks is a situation specific to working with (older?) twisted code. It's still nice to understand what's going on behind the scenes though!
zope.interface.Interface
Twisted makes use of zope's Interface to define a number of abstract interface classes. Speaking personally, I've not seen it used outside twisted, and I think that means it's not supported by much of the typechecking tooling. For example, I've definitely seen PyCharm struggle to realise that it's okay to pass a Response to a function which expects an IResponse! Here's a more complicated example where PyCharm isn't happy with me widening the type LoggingHostnameEndpoint to IStreamClientEndpoint, even though the latter implements the former.
Mypy out of the box doesn't play well with a zope Interface (nor does any other typechecker I tried). Fortunately, the excellent mypy-zope plugin helps here: it teaches mypy that any class like Response which @implements(IResponse) can be passed in place of an IResponse.
Tricks of the trade
At this point I'd like to share a few generic lessons about typing I'd picked up. Nothing ground-breaking here: I think these are all fairly well-known. Hopefully they're the start of a good cheat sheet for annotating—though it pales in comparison to the Mypy cheat sheet.
Annotating decorators
Annotating decorators is fiddly, but it's also vitally important. An unannotated decorator will mask or throw away your decoratee's annotations! The way to write the annotation is best explained by the mypy docs, but briefly: it involves a TypeVar and sometimes a cast too. Here's an example.
Use Callable[..., Any] to describe a generic function with no particular signature.
Use that as a bound on a type variable F. At each usage of @decorator, mypy will deduce a more specific version of F, e.g. Callable[[int], str].
Within that usage of @decorator, F is fixed to that specific type. We use -> F to express that "we return a function with the same signature as the decoratee".
Unfortunately, we don't have a good way to tell mypy that wrapped also has that signature F. We resort to a cast to force mypy to accept this without proof.
Both of these are general types for expressing "I don't know anything about this expression". But only the former will undergo static type checks. We want those type checks to guard against bugs accidentally introduced in the future. For instance, imagine a class with an Any attribute.
from typing import Any
import dataclasses
@dataclasses.dataclass
class C:
label: Any
Imagine in the future we add a new method which assumes that label is a string:
def greeting(self) -> str:
return "My name is " + self.label
Mypy will consider this valid, because no type-checking is done on an Any value. It will complain that it can't prove that greeting returns a str, if --warn-return-any is enabled; but putting that aside, it can't identify the call site as a bug. The bug slips through to runtime.
C(123).greeting() # TypeError: can only concatenate str (not "int") to str
Replacing the Any with object does allow mypy to spot the problem.
error: Unsupported operand types for + ("str" and "object") [operator]
# type: ignore and cast sparingly
There's a good chance that mypy knows better than we do, so we should only overrule it if there's no better option. There are two techniques for this. One option is to tell mypy to just silence the error, by appending a # type: ignore comment to the erroneous line. The other is to force it to accept that a certain expression has a given type: that's what cast is for.
I never like using either of these, but sometimes they're the most practical choice. I'd recommend two best practices for their use, however:
Only ignore a specific error code, e.g. # type: ignore[assignment]. Those codes can be displayed by passing --show-error-codes to mypy.
Leave a comment before every #type: ignore[...] and every cast to justify their correctness. I don't think this is a widely-held practice. I picked it up from the Rust world, where it's encouraged as a way to justify unsafe source code.
There's good stuff in typing
It's worth a read through the module documentation—plenty of things in there I wish I'd known about sooner. There's lots in the toolkit to chose from. Some bits I use fairly often include:
Protocol: lets you formalise duck typing. To use it, define a class that inherits from Protocol. Its methods and attributes are all stubs which describe what you require of objects belonging to this type. It's like an abstract base class or interface, but purely at typecheck time.
Generic: define your own generic types. A bit of a slippery slope to type mania!
[Optional](https://docs.python.org/3/library/typing.html?highlight=typing%20generic#typing.Optional): I use this all the time. It's always good to make your None`s explicit!
NewType: I haven't had a chance to use it much. As I understand it, it's a way to define a "strong typedef". For example, we can use it to distinguish lengths from durations, even if they're both represented by a float at runtime.
I want to call out two parts of typing in particular:
overload is a way to provide extra information about a function depending on how it's called. For instance, consider this function which takes a str or bytes as input and returns its uppercase version.
The problem with this annotation is that we'll have to check at every call site to see if the return value was a bytes or a str object. But we know that uppercasing a str gives us a str, and uppercasing a bytes gives us a bytes. We can use overload to express this.
The first two @overload definitions are like stubs: they're purely used for their annotations. The actual runtime implementation is specified at the end, undecorated. (NB: this specific example is better expressed without overloads, by using AnyStr.)
I was really impressed to see how mypy could use this overloading information. Here's an example:
We can see that f(10) == 1 and so the equality is always False: we'll never print the word "potato". Mypy can reason through this too, if we ask it nicely.
$ mypy example.py
Success: no issues found in 1 source file
$ mypy --strict-equality --warn-unreachable example.py
example.py:16: error: Non-overlapping equality check (left operand type: "Literal[1]", right operand type: "Literal[2]")
example.py:17: error: Statement is unreachable
Found 2 errors in 1 file (checked 1 source file)
I mentioned this in last week's post) when talking about the missing await bug.
Subclassing from TypedDict allows us to define a new type whose values are dictionaries with
a fixed set of keys (some mandatory, some not), and
a fixed type for each key's value.
PEP 589 can best describe the motivation, but in short: there's a lot of source code out there that passes dictionaries around. TypedDict is a way to gradually add typechecking to that code, without having to refactor it to use e.g. a dataclass or a NamedTuple. Let's have a quick example.
from typing import List, TypedDict
class Person(TypedDict):
full_name: str
nicknames: List[str]
age: int
p: Person = {
"full_name": "David Matthew Robertson",
"nicknames": ["dmr"],
"age": 29,
}
Mypy will detect if you delete or omit a required key, or if you insert a key that's not part of the type.
# error: Key "age" of TypedDict "Person" cannot be deleted
del p["age"]
# error: Missing keys ("full_name", "nicknames", "age") for TypedDict "Person"
p2: Person = {}
# error: TypedDict "Person" has no key "eye_colour"
p["eye_colour"] = "brown"
TypedDict is a useful tool, but there are a few important gotchas to be aware of. In my view, these are all minor and worth putting up with, to benefit from the extra type information that a TypedDict provides. Let's take a look.
TypedDict is incompatible with Dict
This one surprised me at first. Why can't I pass my TypedDict to a function that accepts a generic dictionary?
import json
from typing import Dict, TypedDict
class Foo(TypedDict):
bar: str
def print_size(d: Dict[object, object]) -> None:
print(len(d))
f: Foo = {"bar": "baz"}
# error: Argument 1 to "print_size" has incompatible type "Foo"; expected "Dict[object, object]"
print_size(f)
The answer is that it wouldn't be type-safe! For all we know, the function print_size might mutate its argument d. It might add a key, remove a key, or change the type of a value. Each of those would break the contract that TypedDict is supposed to enforce, so mypy is correct to flag this as an error. This GitHub issue has more discussion.
There are few workarounds for this kind of problem.
The function might be able to accept a TypedDict directly.
The function could accept a Mapping instead of a Dict. This is type-safe because the Mapping type does not offer mutating methods such as pop, __setitem__, __del__; but it only works if the function does not mutate the dictionary.
A more drastic approach would discard the TypedDict information with a cast to Dict[str, object] or similar. If so, I'd strongly recommend a comment explaining why the cast is correct and safe.
Mixing mandatory and required fields
Secondly, defining a TypedDict with a mixture of optional and required keys is a little fiddly. All keys can be made optional by passing total=False in the class definition. To have some keys optional and others mandatory, we have to make two TypedDicts. Say for instance we wanted to allow a potentially-missing favourite_colour field to Person. We can't add an annotation favourite_colour: Optional[str] to the first class body. That's optional in a different sense: it would mean that favourite_colour is a mandatory field which is allowed to be None. Instead, we apply total=False to a subclass:
from typing import List, TypedDict
class _PersonRequired(TypedDict):
full_name: str
nicknames: List[str]
age: int
class Person(_PersonRequired, total=False):
favourite_colour: str
This means that a valid Person dictionary may have no favourite_colour key. If it is present, it must be a str.
The syntax is unfortunately a little clunky. PEP 655 acknowledges this and proposes an alternative.
TypedDict is typecheck-time only
One final note: a TypedDict does no validation or conversion whatsoever. If mypy can't know the keys and values a dictionary will have a typecheck-time, you'll need to validate it by hand at runtime. We see this a lot because when deserialising json objects into a dictionary. Here's an example.
import json
from typing import TypedDict
class Foo(TypedDict):
bar: str
# note: Revealed type is "Any"
reveal_type(json.loads("{}"))
# no error. mypy can't do analysis on Any.
f: Foo = json.loads("{}")
Next time
This covers a lot of the machinery and the day-to-day process of annotating Sydent. The last part of this series will give us a chance to quantify our efforts and reflect on the wider typing ecosystem.
Many thanks for reading! If you've got any corrections, comments or queries, I'm available on Matrix at @dmrobertson:matrix.org.
