Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.
Matrix v1.7 has been given a release date of May 25th, right before the next TWIM! Expect a matrix.org blog post with all the details on the day.
Leading up to the release we've seen a number of great spec PRs appearing and being merged! Thank you to everyone for writing them (saving the SCT some time!) and to other reviewing on commenting. It's a huge help and the spec feels like it's chugging along at a blistering pace!
This MSC adds the ability for users who have previously joined a room to rejoin again. Typically this isn't desired in a public room setting, but it does specifically make sense in the case of a DM that you've left and want to return to without the other user needing to invite you. This case has specific implications for cases where there could be only ever one room between two users. Being able to rejoin it if the other user has disappeared is key!
Outside of the DM use case, this functionality can mostly already be achieved by using restricted rooms, where users of a given space/another room can always join your room. However, it would be nice to have the flexibility of allowing certain users to rejoin a room without needing another room to serve as proof of membership.
Is this something you're interested in? Do you have additional use cases? Feel free to check out the MSC and comment with your thoughts!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals/.
The Spec Core Team had another retro this week, and part of what we talked about was how most of the spec PR writing falls to just a few people. Starting this week, we're going to try and diversify that work across the Spec Core Team, both for reducing bus factor and increasing bandwidth of spec PRs. Hopefully the results of that will be noticeable!
For those that don't know, "SAS verification" refers to the act of two devices verifying each other through the use of Short Authentication Strings (SAS). This is the backbone of emoji verification when you verify with another user or one of your own devices.
It turns out that libolm, the olm and megolm encryption library, originally incorrectly encoded the base64 output of the message authentication code (MAC) calculation, which is a values that's passed between devices. Due to trying to maintain backwards-compatibility with older clients, this has not been fixed in newer clients yet.
This MSC proposes bumping the MAC algorithm identifier (which is agreed upon between devices when verifying) to a something new, which allows newer clients to know when to use the new or old (and incorrect) method of calculating the base64 encoding of MAC values.
As far as I can tell, this has no security implications. It's just unfortunate to have incorrect base64 encoding when interfacing with other, non-libolm, implementations of olm and megolm.
The Synapse team is hard at work making Synapse faster and leaner. Work continues
apace on faster room joins over federation, and it seems that the work might land
sooner rather than later, although there are no solid dates yet.
In other news, profiling work is being done to determine ways to hopefully increase
the speed of local room joins and DM creation. Stay tuned for more information in the future on that.
Finally, it was an RC release week. Synapse 1.65.0rc2 was released, and it contains some fun features and bugfixes. Check it out!
This week we released Dendrite 0.9.2 which contains the following updates:
Dendrite now supports history visibility on the /sync, /messages and /context endpoints
It should now be possible to view the history of a room in more cases (as opposed to limiting scrollback to the join event or defaulting to the restrictive "join" visibility rule as before)
The default room version for newly created rooms is now room version 9
New admin endpoint /_dendrite/admin/resetPassword/{userID} has been added, which replaces the -reset-password flag in create-account
The create-account binary now uses shared secret registration over HTTP to create new accounts, which fixes a number of problems with account data and push rules not being configured correctly for new accounts
The internal HTTP APIs for polylith deployments have been refactored for correctness and consistency
The federation API will now automatically clean up some EDUs that have failed to send within a certain period of time
The /hierarchy endpoint will now return potentially joinable rooms (contributed by texuf)
The user directory will now show or hide users correctly
Send-to-device messages should no longer be incorrectly duplicated in /sync
The federation sender will no longer create unnecessary destination queues as a result of a logic error
A bug where database migrations may not execute properly when upgrading from older versions has been fixed
A crash when failing to update user account data has been fixed
A race condition when generating notification counts has been fixed
A race condition when setting up NATS has been fixed (contributed by brianathere)
Stale cache data for membership lazy-loading is now correctly invalidated when doing a complete sync
Data races within user-interactive authentication have been fixed (contributed by tak-hntlabs)
This brings our current test compliance figures to:
Client-server APIs: 90%
Server-server APIs: 95%
As always, please feel free to join us in #dendrite:matrix.org for Dendrite-related discussion.
What? Another week, another major release?
Yes, we want to make breaking changes obvious, and the bridge now requires Node.js 16+.
The release fixes the outdated yarn.lock file. Thanks to the package maintainers of NixOS to point this out! 👋
Furthermore, mentioning Matrix users in Discord got fixed.
And, for the fans of containers, we've re-added the release of Docker images. 🐳
The URL changed and we don't plan to update halfshot/matrix-appservice-discord on Docker Hub.
The image can be pulled from ghcr.io/matrix-org/matrix-appservice-discord:v3.0.0.
Nheko now shows you a pretty preview for rooms you are trying to join. This uses MSC3266, which you might need to enable in your synapse config, if you want to see more than the roomid of the room you are trying to join. It tries to fall back to the /hierarchy endpoint, but that won't work over federation.
Similarly you can now somewhat edit what rooms are in a space. I grew up with Windows 98, so this is a very nested right click menu. We are still discussing some of the wording choices for the options there, so if you have any opinions about it, feel free to give feedback about it! This is part of the bigger goal to be able to manage your communities from within Nheko, so stay tuned for more features coming in that area.
red_sky (nheko.im) also improved our notifications code on macOS. You can now disable the notification sound in the system settings, which for some reason requires badge permissions to show that option... macOS will always remain a mystery to us! Similarly we fixed the source text for notifications on Windows, which should now not show a stray %1 anymore as well as a few other cleanups.
Ever started a DM with someone, only to get distracted and leave them staring into an empty timeline wondering who you are and what you want. Well, it is no more! In this update new DMs will only notify the person you’re messaging once you’ve sent your first message.
That’s not the only exciting update in EleWeb/Desktop this week, including the newly extended voice messages! Instead of voice messages capped at 2 minutes, users are now able to send a voice note of up to 15 minutes long. Go forth and record 🎤
In upcoming releases we’re also looking at how to improve a user’s first few days in Element. We know that it can be daunting to stare at an empty screen and wonder how to get going, which is why we’ve designed and built a checklist for new users. This checklist will help them to get off the ground and messaging friends in no time.
In labs (you can enable labs features in settings on develop.element.io or on Nightly):
Threads improvements are still very much underway. We have a proof of concept (PoC) for improving notifications that we’re testing to ensure scalability and performance.
Our release this week was rejected by Apple. Their feedback? Our info.plist comments do not provide enough context as to why we’re asking for access to things like the camera, photos, or contacts. Never fear! We’ve updated our copy and we’re confident that it’s clearer than ever.
Also in the upcoming release:
Fixing the crashes that some users were running into when switching space.
The new in-app notifications now also appear in the notification centre
And a bug fix for the rare issue of sending duplicate images to a timeline when sending multiple photos
We’re also working on changing the layout of our mobile apps, and work is well underway on iOS. We’re excited to share these big changes with you so keep your eyes peeled on our socials and in other TWIM notes.
Join us over at #element-community-testing:matrix.org on Tuesday 16th August at 12:00 BST to try out the new app layout on iOS! There’s a little bit of setup required so check out the room ahead of time for instructions :)
In the upcoming Android release we’ve fixed some crashes; including where the app would crash if a user attempted putting non-ASCII characters in their MXID during account creation.
Biometric login is now disabled if the device you’re on does not support biometrics.
We’ve also been working hard on adding unit tests to increase coverage.
There’s work being done on the modularisation of languages that may help decrease the complexity of translation and move us towards a set of common strings between platforms.
Join us over at #element-community-testing:matrix.org on Tuesday 16th August at 15:00 BST to try out the new app layout on Android! There’s a little bit of setup needed so check out the room ahead of time for instructions 🙂
v2.1: Custom emojis and stickers (Birthday edition)
Hello everyone,
On July 28 project marked its one-year milestone. During this time it has grown at an unexpected rate, both in terms of development as well as popularity. This is great news, so let's celebrate that with this birthday edition.
In this update, we have added Custom emoji and sticker support to Cinny. Custom emoji and stickers were not in our roadmap but that's the surprise. Aren't they cool?
Apart from the emoji and stickers, there are tons of other features such as Blurhash, Mark entire space as read, user pills, and bug fixes. Check out the releasepage for a detailed changelog.
In the case of the roadmap, we are still on that, work on the "Rich input editor" is in progress and will come as the next release. Stay tuned for that!
A new version of the Ruby Matrix SDK has now been released, adding a new abstraction in the form of a Sinatra-inspired bot DSL, as well as general fixes and improvements. Making a Ruby-based Matrix bot can now be as simple as;
#!/usr/bin/env ruby
require 'matrix_sdk/bot'
command(:praise, desc: 'Gives you praise', only: -> { room.user_can_send? client.mxid, 'm.reaction' }) do
room.send_notice "#{sender}, you tha' man!"
room.send_event 'm.reaction', {
'm.relates_to': {
rel_type: 'm.annotation',
event_id: event[:event_id],
key: '👍️'
}
}
end
Next weekend, Sat 20th and Sun 21st, FrOSCon will be taking place. It's a conference about open source software near Bonn, Germany.
There will be some program around Matrix, though I'm still looking for volunteers to help me with a stand and DevRoom.
For the stand we want a small group to explain Matrix to others. For the DevRoom we want some talks and workshops. Anyone's welcome to ask me for more info or to get involved.
Announcing matrix-locust, a new tool for load testing Matrix homeservers, based on the Python load testing framework Locust. It's very early days for this project, but we're already discovering some interesting results. There isn't an official tagged "release" yet, but anyone interested in this topic is encouraged to stop by #load-testing:matrix.org and say hello.
Have you ever felt lost in the Matrix world? Too many rooms and spaces to manage? Well, back by popular demand (with Timo's blessing), I present, The Room of the Week! Every week we strive to highlight a room or a space that we believe deserves attention for discussing interesting going on across the Matrix Network.
A place to discuss buying, brewing, and drinking everyone's favorite morning beverage of choice! I had no idea there was so much Nuance to this topic until I joined this room . For example, did you know there was a correct way to distribute coffee grounds when you first pour to ensure maximum flavor? Neither did I! Whether you are new to the world of coffee drinking, or a seasoned connoisseur of morning Joe, this room has something for everyone who loves coffee.
If you have a room you wish to see highlighted, join us at:
https://matrix.to/#/!bIyiUUnriVoHtYzuPS:fachschaften.org?via=chat.shawnsorbom.net&via=matrix.org&via=fachschaften.org
to get your favorite room of the week highlighted.
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
This week saw the merge of three MSCs from first-time MSC authors! MSC3818 (Copy room type on upgrade) from @Mikaela and both MSC3786 (Add a default push rule to ignore m.room.server_acl events) and MSC3827 (Filtering of /publicRooms by room type) from @SimonBrandner. Nice work to both of them!
A bundle of small quality-of-life changes to the .well-known server and client endpoints. Removing some ambiguities, replacing old and deprecated endpoints as well as a potential size cap suggestion.
Looks like a good contender for one to push over the line given a bit of review and response!
TWIM: someone in #matrix:matrix.org complained that the image in the spec section is always the same, so I made this gif (actually webp) out of all those images:
We've started work on our HG HomeServer written in pure TypeScript, compilable as a single JS file, with no dependencies except NodeJS. It's intended for a special use cases when Matrix is used as a backbone for custom apps. It's lightweight, minimal and for the moment isn't even planned to support full Matrix spec. We might make it possible to run it on browser later. https://github.com/heusalagroup/hghs
Another week, another release! Synapse 1.64.0 was released this week, featuring
a host of new features, bugfixes, and internal changes aimed at reducing memory usage,
increasing performance, and improving the developer experience. Check out the full list of changes
here. In addition, work continues on faster room joins. The goal gets closer every day!
This week we released Dendrite 0.9.0 and Dendrite 0.9.1. There are quite a few big changes, including an all-new caching model and several optimisations. This release also moves our baseline supported Go version up to Go 1.18.
The following changes are included across both releases:
Dendrite now uses Ristretto for managing in-memory caches
Should improve cache utilisation considerably over time by more intelligently selecting and managing cache entries compared to the previous LRU-based cache
Defaults to a 1GB cache size if not configured otherwise
The estimated cache size in memory and maximum age can now be configured with new configuration options to prevent unbounded cache growth
Added support for serving the /.well-known/matrix/client hint directly from Dendrite
Refactored membership updater, which should eliminate some bugs caused by the membership table getting out of sync with the room state
The User API is now responsible for sending account data updates to other components, which may fix some races and duplicate account data events
Optimised database query for checking whether a remote server is allowed to request an event over federation without using anywhere near as much CPU time (PostgreSQL only)
Database migrations have been refactored to eliminate some problems that were present with goose and upgrading from older Dendrite versions
Media fetching will now use the /v3 endpoints for downloading media from remote homeservers
HTTP 404 and HTTP 405 errors from the client-facing APIs should now be returned with CORS headers so that web-based clients do not produce incorrect access control warnings for unknown endpoints
Some preparation work for full history visibility support
Upgrades a dependency which caused issues building Dendrite with Go 1.19
The roomserver will no longer give up prematurely after failing to call /state_ids
Removes the faulty room info cache, which caused of a number of race conditions and occasional bugs (including when creating and joining rooms)
The media endpoint now sets the Cache-Control header correctly to prevent web-based clients from hitting media endpoints excessively
The sync API will now advance the PDU stream position correctly in all cases (contributed by sergekh2)
The sync API will now delete the correct range of send-to-device messages when advancing the stream position
The device list changed key in the /sync response should now return the correct users
A data race when looking up missing state has been fixed
The /send_join API is now applying stronger validation to the received membership event
Fixes a crash that could occur during event redaction
The /members endpoint will no longer incorrectly return HTTP 500 as a result of some invite events
Send-to-device messages should now be ordered more reliably and the last position in the stream updated correctly
Parsing of appservice configuration files is now less strict (contributed by Kab1r)
The sync API should now identify shared users correctly when waking up for E2EE key changes
The federation /state endpoint will now return a HTTP 403 when the state before an event isn't known instead of a HTTP 500
Presence timestamps should now be calculated with the correct precision
A race condition in the roomserver's room info has been fixed
A race condition in the sync API has been fixed
As always, please feel free to join us in #dendrite:matrix.org for more Dendrite-related discussion.
This week has seen the usual updates to my Helm Charts - with element-web being updated to 1.11.2 and matrix-synapse to 1.64.0.
Additionally, the matrix-synapse chart now also allows for adding entirely custom .well-known data - along with an example on how to use that for MSC1929.
This bridge connects users on Matrix and Discord – or other platforms, if combined with other bridges.
Earlier this year the community bridge has been adopted by the Matrix.org bridge team to give it some attention.
Its most recent update dated back to December 2020 and some fixes waited for a new release.
Well, here it is! v2.0.0
Its breaking changes are the requirement of NodeJS 14 or newer and the usage of yarn instead of npm install.
Furthermore, the update introduces a changelog and rolls out the guidelines we use for developing other matrix.org bridges.
„Start DM only on first message“ has landed on develop. Changing the DM flow from invite → message to message & invite at the same time. If you want to see it live, join our testing session at 12:00 BST on Tuesday
Version 1.8.24 is available on the App Store with our new Sign Up and Sign In flows. The update is rolling out slowly and should be available to everyone by Monday.
The work on our new app layout is coming along nicely with much of it merged into the repo (but disabled behind a build flag).
In-app notifications will now also be delivered to Notification Centre.
Continuous improvements are being made to the Live Location sharing feature.
Integration tests are now run for every PR on matrix-ios-sdk more than doubling the reported test coverage!
1.4.31 is rolling out which includes the new and improved FTUE onboarding experience along with fixes for markdown lists no longer always starting from 1 and html entities showing up in messages.
The new app layout is starting to materialise with PRs available on github for anyone interested in having a sneak peek!
We’re continuing to make improvements to Live Location sharing and cross signing verification as well as investigating performance issues.
📰 You want updates more frequently and close to when they actually happen? Join our Effektio Matrix News room, discuss general aspect in our foyer or hang out with the devs in our tech channel.
After a short vacation, I've done some new work on populus-viewer. Most of this has been UX and bugfixes, but I've added one new feature that I wanted to share. MSC3775: Markup Locations for Audiovisual Media gives a spec for annotating audio and video on matrix, but it also allows you to annotate images. So, for completeness sake, I've added support for annotating image files in Populus! This might be useful for discussing publication layouts, product designs, or for teaching art history. So populus can now annotate: video, audio, images and pdfs.
As always, if you want to learn more, follow populus development, or discuss the future of decentralized social annotation on Matrix, come join us at #opentower:matrix.org.
matrix-widget-api v1.0.0 was released yesterday, reflecting the fact that we're not expecting any big changes to the library's architecture for the foreseeable future, and that it's more or less ready for wider use.
It also comes with a couple of new features ✨ for widget authors: Sending and receiving to-device messages with MSC3819, and getting TURN servers from the client with MSC3846. Together, these features enable some new complex use-cases, such as doing VoIP from inside a widget. See Matrix Live for a preview of what that looks like with Element Call! 📹️
This is as good a time as any to mention that matrix-widget-api doesn't have to just be for web-based apps. If you're building mobile apps with Matrix and want to make use of widgets, running matrix-widget-api inside a web view to liaison with the actual widget can make it a lot simpler to start supporting widget API features. If you're curious, watch Element iOS+Android for upcoming examples of this.
This week we've been building on the initial work on the pion call server (SFU) from Sean, and have made our very first call through it with Element Call! It's still very early and there's still lots of work to do, but this will allow Element Call to scale up to much higher numbers of people.
The VoIP team are also looking after widgets now, and in our quest to embed Element Call into the Element apps, we've hit the milestone of releasing matrix-widget-api 1.0.0. We're also making great progress towards embedding into Element Web.
This was also a week of fighting the ci. After the integration tests landed, coverage reporting broke and an investigation was kicked off to check whether llvm-cov is in a usable state for us by now (it is not), but a fix was found. We've also started regretting adding npm-based workflows in our repo, as we've found ourselves at the end of an upgrade bug and the saw CI failing without us changing anything :shakes_fist_at_sky: . A fix was found quickly by pinning a sub-dependency. On a similar note, we've postponed merging the kotlin bindings until the android team is back on it and available to answer some questions we have, and the wasm-js bindings showed build failures on CI, which the original author will have to take a look at after coming back from vacation - thus delaying crypto-js a bit further.
👉️ Wanna hack on matrix rust? Go check out our help wanted tagged issues and join our matrix channel at Matrix Rust SDK.
It's been a while since I've made an update on Polyjuice Client Test. Since the last update:
several more tests have been added
there have been some some front-end improvements
I've also improved the documentation, done some refactoring, and added some helper/utility modules to make things clearer. My goal is to make it easier for people who don't know Elixir to be able to read and write tests without running away screaming.
compatibility with some Matrix clients has been improved
added some functions to make it easy for tests to completely override Matrix endpoints
I'm also happy to say that Polyjuice Client Test has helped find bugs in some clients, which have since been fixed.
This week I'm showing off an early look at the Matrix public archive. As the name suggests, it acts as an archive of history for your world-readable Matrix rooms. This allows you to view historical content day-by-day and jump back years ago to see what your Matrix room was up to.
More importantly, it also allows Google to do the same thing so you’ll probably start finding Matrix content from your favorite search engine and be able to harness the massive knowledge base stored in Matrix. Imagine seeing Matrix logs instead of Stack Overflow answers when googling questions! The new portal into the Matrix ecosystem 🌌
Under the hood, we use the MSC3030 /timestamp_to_event endpoint to fetch the messages for a given day and then we sever-side render the events with the Hydrogen SDK. Re-using Hydrogen gets us pretty and native(to Element) looking UI and keeps the maintenance burden of supporting new event types in Hydrogen.
If you want to follow what’s going on and see how it's coming along, you can checkout the project on GitHub, https://github.com/matrix-org/matrix-public-archive
I was able to to host a matrix-synapse server on termux app on an android device. I documented the process on termux-synapse github repo And started working on a script that automates the process (final commit is being currently tested before pushing).
It was a "for fun" type of thing. But I can see it being useful for people who do not have access to a raspberry pi (such as myself at the moment) to use as a small homeserver. It can hold up in 1 to 1 Direct Messages and in small rooms.
Have you ever felt lost in the Matrix world? Too many rooms and spaces to manage? Well, back by popular demand (with Timo's blessing), I present, The Room of the Week! Every week we strive to highlight a room or a space that we believe deserves attention for discussing interesting goings on across the Matrix Network.
A matrix space dedicated to finding all of the free open source games, engines, and assets in the Matrix world so that you don't have to. Helpfully organized, and well maintained, it is the Premier stop for open source gaming on The Matrix Network!
If you know of a room that you would like to see highlighted, please visit
https://matrix.to/#/!bIyiUUnriVoHtYzuPS:fachschaften.org to let us know of the room that you would like to spotlight.
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
