Gajim - 2025-02-23

y does gajim add \uFE0F after every emoji?

https://downloadable.pain.agency/file_share/2N7U7KjfC-gHalj0rY_ksmB-/6d087055-de66-4c96-88fa-dcc730e1b119.png

whered the button go 🤔

it works if i click the empty space there, but theres no button displaying

hau: it's called a emoji variant selector, what's the problem with it?

lovetox, it was causing an issue with reactions for my bot, cause it wouldn't recognize the emoji as valid when the message came from gajim. ``` reaction.set_value(reaction_txt, all_chars=all_chars) File "/usr/lib/python3/dist-packages/slixmpp/plugins/xep_0444/stanza.py", line 55, in set_value raise ValueError("%s is not a valid emoji" % value) ValueError: 🍫️ is not a valid emoji ``` I fixed it now by just filtering the character out, but just wondering if there's any purpose to it since other clients don't do it.

depends there are emojis where it is valid, if you use gtk3 they added the variant to all emojis which is technically not valid for all emojis

but you are talking about reactions or?

i thought we filtered out all variant selectors

we remove all variant selectors

https://dev.gajim.org/gajim/gajim/-/blob/master/gajim/common/modules/reactions.py#L176

and im sure this is also in the stable release

so i would doubt that you received this from a current Gajim release

either way, you need to be prepared to receive them, they are only invalid for some emojis not for all, this is defined in the emoji standard

no not from reactions, from messages (where the bot gets the emoji to send as reaction from)

> depends there are emojis where it is valid, if you use gtk3 they added the variant to all emojis which is technically not valid for all emojis do u have any examples of emojis where it is valid? ↺

but that makes not much sense, i can send in a message what i want, not just valid emojis :D

https://www.unicode.org/reports/tr51/#Emoji_Variation_Selector_Notes

and here examples

https://www.unicode.org/reports/tr51/#Emoji_vs_Text_Display

> but that makes not much sense, i can send in a message what i want, not just valid emojis :D well sure, and when an invalid emoji is sent it won't do anything, it's just weird that gajim sends invalid emojis when picked ↺

Gajim still crashes if the funny unicode string is present in, for example, MUC topic

voker57: in nightly?

in Debian Experimental, at least

No idea what version they ship

1.9.5+git20250208.1b5501f8-1

This fix is included since Jan 9th https://dev.gajim.org/gajim/gajim/-/commit/1cece0fc0ee1e5eaaa7ba936a72046d89d0967ce

voker57, where does it crash? when you join the room, or when you open the chat details dialog?

when I open the chat details

chat itself does not crash after updating to experimental

yes seems we forgot to add the fix there

lovetox pushed 1 commits to branch gajim/master cfix: Add pango word break workaround in more places - https://dev.gajim.org/gajim/gajim/-/commit/e5dc6e72cdf221967e96114781850ed13fb420d4

thanks for reporting

lovetox pushed 1 commits to branch gajim/master change: Require Python 3.11 - https://dev.gajim.org/gajim/gajim/-/commit/60f3cc5c984107941723bd382442fb0f4931e730

lovetox pushed 1 commits to branch gajim/master cfix: Multiprocessing: Limit max tasks - https://dev.gajim.org/gajim/gajim/-/commit/6bceac5eb8913b45a53e96d07a646b207204de83

wurstsalat pushed 1 commits to branch gajim/master cq: Iq module: Remove strong type assertion - https://dev.gajim.org/gajim/gajim/-/commit/710d772e777ad20c3d98940422d45215e036bf24

lovetox pushed 1 commits to branch gajim/master refactor: Use shorter version for UTC timezone - https://dev.gajim.org/gajim/gajim/-/commit/5ad6b44e5e58e5da33dc96a25fbbab0bac039350

https://superfastpython.com/processpoolexecutor-max-tasks-per-process/ > Unexpected Behavior With Maximum Tasks Per Child Process I wonder if this could be an issue also affecting us, can check it later.

hm yes seems i can reproduce what that blog is saying

seems like a bug, i found no bug report in python about it

https://github.com/python/cpython/issues/115634

nice that you found this

3 days ago > I think we should try to make some progress on this PR, it solves a real issue which is reported by multiple users. Hopefully the fix will land soon https://github.com/python/cpython/pull/115642#issuecomment-2673024916

lovetox pushed 3 commits to branch gajim/master cfix: Mutliprocessing: Disable max limit of tasks - https://dev.gajim.org/gajim/gajim/-/commit/8ba034a19563775be179bbd89300f4016e1eaa22 cq: Update ruff config and fix errors - https://dev.gajim.org/gajim/gajim/-/commit/6686f97ac6fb7f2b62c49463b69cec1cb9bbc8c4 ci: Update ruff to 0.9.7 - https://dev.gajim.org/gajim/gajim/-/commit/42413fbcb2cd0f91a3cff4560f30b97791e0970c

i dont think this is a big problem

this just means that on long running Gajim instances we probably have more memory consumption

but as long as we just use it for now for thumbnails, i dont think this will be noticable

and to fix that Gajim just has to be restarted

Maybe some naive question: Where would we expect to accumulate memory over time which can't be freed but require a restart and why can't it be freed without a restart respectively when is getting ref count to zero and gc not possible?

in the worker process

no application is free from memory leaks

> no application is free from memory leaks unless you write it in rust! /s ↺

this explains why i was getting huge memory consumption on my long running gajim client for weeks

r00tobo: if you're referring to leaks, yes. The change being discussed is only some days old

> unless you write it in rust! /s No, Rust protects from other kinds of memory errors. It'll help with untracked leaks (at least some some degree), but tracked leaks are still a problem.

> /s

mesonium, lovetox, instead of doing your own thumbnailer, are you aware of Glycin which is used by GNOME’s Loupe to decode images in a sandbox, as to avoid exploits as much as possible?

betarays, glycin does both sandboxing (using bwrap) and uses the memory safe image crate, for security in depth.

oh, nice

maybe it’s not the same image crate as the one I stumbled upon some time ago (I think it’s the one used by servo), but it was still using other libraries to do the decoding

There are some memory-unsafe libraries being used optionally, for instance if you want AVIF support it will use dav1d (which is written in C and assembly).

But there is no other option for this format, if you want it to be performant you kind of have to use assembly.

oh, AV1 is that hard to decode that you _need_ extreme optimization or hardware decoding?

I wouldn’t call that extreme, but yes you generally want optimizations to decode image data.

sure, but compilers already apply some level of optimizations

Hardware acceleration is one area which glycin currently can’t do, due to the sandbox it won’t be able to access the required /dev/video* and /dev/media* devices.

I guess that makes sense

betarays, for image data, you generally want to use the SIMD units in your CPU, which let you handle 16, 32 or even 64 times as many data as with scalar code.

Yes, I’m not sure whether compilers are any good at outputting SIMD instructions

Compilers try, but unless they recognize the exact pattern being used they generally give up at auto-vectorization and end up producing scalar code.

They are good at what they have been written for, simple arithmetic loops.

though presumably you could reduce your inline assembly enough to be trustworthy enough

You could technically write an auto-vectorizer able to generate the same code as dav1d’s and ffmpeg’s asm decoders, but that would basically amount to pattern matching their exact functions, at which point you’re generally better off writing the assembly yourself.

betarays, that’s what dav1d did.

but the rest of dav1d is C, right?

The C code contains the generic functions it needs on every platform, and the assembly is for equivalent functions, which are generally self-contained algorithms.

I mean, I opened their repo and it seems the C code is extremely small compared to the assembly

You also have https://github.com/memorysafety/rav1d which rewrites the C parts into Rust.

cool, I only knew of rav1e

I wouldn’t say extremely small, it’s enough to get a fully functional AV1 decoder on any platform, the assembly parts are specific to each platform.

rav1e is a completely different project, it is an encoder instead of a decoder, but it includes a few of the dav1d functions to optimise the decoding parts of the encoder.

I meant that gitlab says it’s 20% and most of the rest is assembly

(but of course the assembly has to be written for all platforms so…)

(Modern codecs rely on decoding during encoding, in order to estimate the error rate you got and do a bunch of other targeted encoding decisions.)

betarays, also most assembly languages are much more verbose than the equivalent C code, so if GitLab is counting per line it’s likely to inflate its weight.

right

Works only on linux

I see no python bindings

Making Python bindings sounds easy enough.

This just loads images it has no functionality to make a thumbnail

Pillow can load all files formats under the sun and provides extensive image utilities and tools

What is the benefit here? That you can exploit it less because it's sandboxed and in rust?

Yes.

lovetox pushed 2 commits to branch gajim/master refactor: Replace deprecated getdefaultlocale() - https://dev.gajim.org/gajim/gajim/-/commit/0cfc21afbc37db8058aed8492a88af01fa0c1965 refactor: Replace deprecated Widget hide() and show() - https://dev.gajim.org/gajim/gajim/-/commit/d8ef58f4f641d5ea44fdd52b2fcb8a321435358a

lovetox pushed 1 commits to branch gajim/master refactor: Dialogs: Simplify showing dialogs - https://dev.gajim.org/gajim/gajim/-/commit/93c1ea1d7381b59bd835a4cb57c80242cc9b86fb

wurstsalat pushed 1 commits to branch gajim/master cfix: ServerInfo: Fix checking for SM; improve type annotations - https://dev.gajim.org/gajim/gajim/-/commit/a8e22ed906333a5516c262ba6afbe1d796ee585a

lovetox pushed 1 commits to branch gajim/master cfix: MUC: Fix assertion - https://dev.gajim.org/gajim/gajim/-/commit/f78ad7c421519172dde100b82e68693012e13f3b

had to restart gajim because it froze out on me

wurstsalat pushed 1 commits to branch gajim/master cq: StartChat: Remove GTK4 TODO - https://dev.gajim.org/gajim/gajim/-/commit/8e9d8f49be2fcd4762fe216e20b345471e62d787