Gajim - 2018-02-21

  1. Daniel lovetox, to edit the wiki, do I have to clone it or is there a way to edit it online?
  2. lovetox Daniel, online but i have to add you to the Project
  3. lovetox Done !
  4. lovetox you should see now an edit button the wiki pages
  5. lovetox btw i found the issue why PMs show double
  6. Daniel great, thank you! :)
  7. lovetox muc pms are just a fucked up thing in xmpp, short version is we have to deduplicate
  8. Daniel ah I see
  9. lovetox the muc server sends the message to every resource of yours that is joined the muc
  10. lovetox and afterwards your server does a carbon copy and sends the message again
  11. Daniel hm ok, and deduplication for these messages, is it easy to implement?
  12. Maranda *Ponders* why do you have to dedup lovetox? 🤔🤔
  13. lovetox yeah we do it already somewhere else, but i never had reason to do it for carbon copys
  14. lovetox its 2 lines nothing special :)
  15. lovetox Maranda, i just wrote why some lines above
  16. lovetox MUC forks message to all clients
  17. lovetox Server sends carbon copys to clients for message
  18. SouL Wasn't something like complex servers, simple clients?
  19. SouL :D
  20. lovetox Yes but this does not apply for muc pms
  21. lovetox muc pms you have to not trust any server or standard
  22. lovetox and just dedup on origin-id and stanza-id
  23. lovetox :)
  24. Maranda The server sending it to every resource of yours... It should send it only to the most available reasource in case of nick merge, it should fork only if carbons is activated
  25. Maranda Sounds like buggy implementations
  26. Maranda To me.
  27. lovetox Maranda how would a server know if my client has carbons activated?
  28. Maranda Your server does :P
  29. lovetox you realise that a muc can be on another server
  30. Maranda But the muc service shouldnt send to all resources on a merge
  31. Maranda Only the most available, just saying
  32. lovetox there is basically no correct way
  33. lovetox every server does there own thing, and every thing has cons
  34. Maranda Yed
  35. lovetox Holger posted a funny link about that yesterday
  36. lovetox i try to find it
  37. Maranda But it's your server that does the carboning
  38. Maranda Not the muc service
  39. lovetox
  40. Maranda Problem is always nick merges because it's implementation discretional there's no viable directions in the xep (as usual)
  41. lovetox i dont care anymore, as i said i dedup on origin-id and stanza-id, and servers and mucs can do what they want
  42. SouL Pragmatism!
  43. Maranda Read the article, I keep track of directed presences and send only to the most available resource as long as carbons isn't activated then I fork
  44. Maranda Keeping track of directed presences is not that hard
  45. lovetox if you say so i believe you, but as client dev im not having anything from one server doing it right :)
  46. Maranda 🤣 🤣 🤣
  47. lovetox i have always dev for the weakest link
  48. Maranda Haha
  49. Maranda And again everything could have it's quirks still, MUC is basically a big hack after all
  50. Maranda (it's by today still the module with the most lines of code in my codebase)
  51. Maranda (iirc)
  52. Ge0rG hey lovetox, did you figure out the MUC-PM issue?
  53. Ge0rG BTW, I still had an annoying popup from Gajim yesterday, about a contact accepting my subscription request.
  54. Holger Simply tagging MUC PMs as such solves most of these issues.
  55. Holger Maranda: Oh you don't fork PMs to all MSN sessions? That's different from Prosody, right?
  56. Ge0rG Holger: simply tagging PMs doesn't solve any of the issues.
  57. lovetox Ge0rG, and i guess you find this information not useful?
  58. Ge0rG I've updated now
  59. lovetox because this is nothing new, gajim does that since ancient times
  60. Ge0rG lovetox: it is useful, but not in an "annoying popup useful" way
  61. Ge0rG annoying popups should be reserved for the apocalypse
  62. Holger Ge0rG: Well if the forking behavior differes between servers then I'm back to "just give up and ditch PMs".
  63. lovetox yeah but this is currently the concept, events are shown in the roster and when you click on it they are opened
  64. Ge0rG lovetox: but I didn't click and it showed a popup?
  65. Holger Otherwise the <x/> mark does of course allow the carbons code to avoid copies for incoming messages. Outgoing is still a PITA. So yes "solves most of these issues" was a bit optimistic :-)
  66. Ge0rG Holger: maybe we can scare Maranda into fixing it, or just ignore it because nobody uses that server anyway.
  67. lovetox Holger, tagging muc pms with muc#user, only solves that i dont have to do a disco on the jid if i get a carbon from a muc im not joined
  68. Ge0rG Holger: I've just transferred my master plan on how to actually fix all the issues into the wiki
  69. Holger lovetox: As I said the carbons module can then avoid generating duplicates in the first place.
  70. lovetox ah you meant server side
  71. Holger lovetox: Both Prosody and ejabberd do that for ages, IIRC>
  72. Holger lovetox: Yup.
  73. lovetox yeah i only thought about what i do with this information
  74. Holger Ge0rG: Cool, I'll reload.
  75. Ge0rG lovetox: please reload and read
  76. Ge0rG :)
  77. lovetox reloading
  78. Ge0rG it's short enough, and it tells you exactly what to do
  79. Ge0rG lovetox: also don't listen to Maranda
  80. Holger Ge0rG: Right, that's how I implemented things.
  81. lovetox hm Ge0rG i dont agree with droping carbons
  82. lovetox hm or yeah
  83. lovetox i get them later through mam
  84. lovetox but hm no i could store them in this moment
  85. Ge0rG lovetox: which carbons?
  86. lovetox why drop them and get them later
  87. Holger lovetox: What?
  88. lovetox the ones from mucs im not joined
  89. Ge0rG lovetox: MUC-PMs shouldn't ever land in MAM
  90. Holger lovetox: You were fighting with dups and Ge0rG tries to avoid them, no?
  91. lovetox hm but they do or not?
  92. Holger Ge0rG: Hehe that's the next fun topic.
  93. Holger Ge0rG: So they remain unreliable.
  94. Ge0rG Holger: you can't fix them anyway.
  95. Ge0rG Holger: do you want them in MUC MAM? In account MAM?
  96. lovetox my last info was, muc pms go into mam
  97. Ge0rG lovetox: into which mam?
  98. lovetox user
  99. lovetox account
  100. Ge0rG Holger: what do you say?
  101. Holger Ge0rG: It's my part to make the statement that PMs are unfixable. You usually respond with "of course you can, you're just lazy!".
  102. Holger Ge0rG: Who are you and what have you done to Ge0rG?
  103. Holger Ge0rG: I have no idea what to do about MAM vs. PMs.
  104. lovetox Ge0rG, muc pms go since ever into account mam, daniel just proposed a month ago they shouldnt
  105. lovetox but i dont think there was an agreement
  106. Holger Yes they do, question is how it *should* be done.
  107. Ge0rG lovetox: no, Daniel proposed not to put normal MUC messages into MAM
  108. Holger ... which doesn't happen anyway with Prosody/ejabberd. But the XEP says it SHOULD happen.
  109. lovetox hm i think you are right, maybe im not remebering this correctly
  110. Holger Kev likes the idea.
  111. Ge0rG Holger: your statemens are full of unresolved references.
  112. lovetox either way the problem is not with carbons of mucs im not joined, they dont lead to duplicate messages
  113. Holger Ge0rG: [citation needed]? Is this Wikipedia?
  114. lovetox the problem was when im joined with 2 resources, the muc forks the message to all resources
  115. Ge0rG lovetox: yes.
  116. lovetox and afterwards the server sends carbons out for them
  117. Ge0rG lovetox: yes.
  118. Ge0rG lovetox: you can ignore those carbons
  119. lovetox the only way is deduping on origin-id / stanza-id
  120. lovetox Ge0rG, can i?
  121. Holger Or more ugly de-dup magic.
  122. Ge0rG lovetox: just ignore received carbons for MUC-PMs
  123. lovetox is it a MUST in the xep that ever muc has to fork to all resources
  124. lovetox ?
  125. Ge0rG whoops.
  126. lovetox i dont think so
  127. lovetox as a client i can not know if muc forks to all resources, and because of that i can drop all carbons
  128. Ge0rG I just realized I did it wrong in
  129. Ge0rG lovetox: all sane MUC implementations will fork PMs
  130. Ge0rG lovetox: because the MUC can't know whether your account has Carbons enabled.
  131. Ge0rG account or client
  132. lovetox ok so you are saying, drop received carbons
  133. Ge0rG lovetox: exactly
  134. Ge0rG lovetox: that's how yaxim is doing it for a year now.
  135. lovetox so why do server send carbons if all sane servers fork it themself?
  136. lovetox and yes muc#user was added in my yesterday example
  137. Holger Ge0rG: What have you done wrong there? Looks good to me ...
  138. Ge0rG lovetox: because server and MUC are two different entities
  139. Ge0rG Holger: I just fixed it. client handling of "received" carbons when joined was wrong
  140. lovetox yeah and? my server knows this is a pm, it knows all sane server send it to all resources, so why sending out carbons
  141. Daniel is confused, thinks they mean the other daniel
  142. Ge0rG Daniel: it depends on which daniel you are.
  143. Holger lovetox: Current servers *don't* generate carbons for MUC PMs. The rule is just to cope with servers that do.
  144. Ge0rG lovetox: your server can't know if it is a MUC-PM if it's not <x/> tagged
  145. Daniel Ge0rG, not the Gultsch one :D
  146. lovetox but it was in my example, to be honest i dont know on which server i tested this
  147. Ge0rG Daniel: in that case we are talking about the other daniel, sorry.
  148. lovetox when did ejabbered add this?
  149. Holger lovetox: Maybe the PM wasn't tagged with <x/>.
  150. Holger lovetox: Quite a while ago, I can check Git ...
  151. lovetox ah no,
  152. lovetox the client has to add the x
  153. lovetox when sending
  154. Holger The MUC service adds it.
  155. Holger The client should also add one yes.
  156. lovetox im not sure gajim does that
  157. Ge0rG Holger: will ejabberd carbon-copy "sent" messages containing an <x/>?
  158. Holger Ge0rG: I realized that I forgot implementing this ~10 minutes ago.
  159. Holger Well I forgot pushing it. does :-P
  160. lovetox but client adding x is only to not trigger sent carbon
  161. lovetox my thingy was with received carbons
  162. Ge0rG lovetox: but we need sent carbons!
  163. Ge0rG I've just removed the <x/> tag requirement from the client impl.
  164. lovetox so why add x? if we need sent carbon, server should treat it like every other message i send
  165. Ge0rG when sending a MUC-PM, the client should ~tag it with <x/> (to let their server know)~ (this might prevent Carbon-copying to other clients), but not with <private/> (to allow sync to the user's other clients)
  166. lovetox my head hurts
  167. Holger Ge0rG: Hmmm.
  168. Ge0rG lovetox: broken server implementations might stop CCing your "sent" message to your other clients if it contains an <x/>
  169. Ge0rG Holger: is your server broken?
  170. lovetox so why would i add it Ge0rG
  171. lovetox i think i want that its carboned to my other clients or not
  172. Ge0rG lovetox: yes, I've just changed the wording in the wiki. Sorry for the confusion.
  173. Holger Ge0rG: Without <x/>, the other clients can't follow the rules when receiving a "sent" Carbon, no?
  174. Ge0rG lovetox: yes you want. Don't add <x/>
  175. Holger I'm not sure :-)
  176. Ge0rG Holger: without <x/> the other clients need to disco#info the JID
  177. lovetox ah
  178. lovetox yeah it was about the not joined thing again
  179. lovetox now i get it
  180. Holger Ge0rG: As I said above, I forgot to push the change to adhere to that rule, yes. That rule was the most recent one IIRC :-)
  181. Ge0rG Holger: so ejabberd is broken. Good to know
  182. Holger Ge0rG: ...
  183. lovetox so generally i want to add it, but we just found out that this als triggers carbons which in turn we dont want
  184. lovetox am i seeing this correctly
  185. Ge0rG Holger: not criticizing you, just trying to figure out the best way to move on
  186. Ge0rG lovetox: sorry, no.
  187. lovetox what !
  188. Holger Ge0rG: Since your last wiki edit, ejabberd is completely fine again!
  189. Ge0rG lovetox: generally you want to add it, but it triggers the server to NOT send carbons, which we DO want
  190. lovetox ah yeah
  191. lovetox ok
  192. Holger Ge0rG: "Broken" or not depends on which revision of that wiki page you look at.
  193. Ge0rG Holger: my last wiki edit is a workaround for broken servers
  194. Holger Ge0rG: I wouldn't add such workarounds if they have downsides.
  195. Holger Ge0rG: And this one has, I think.
  196. Holger Ge0rG: Requiring the client to disco#info sucks, IMO.
  197. Ge0rG Holger: thank you very much for pointing this ous
  198. Holger Ge0rG: If that requirement is fine there's no point in the <x/> tag.
  199. Holger Ge0rG: Sarcasm? I can just shut up.
  200. Ge0rG Holger: luckily we can completely skip disco#info because "received" carbons are guaranteed to come from a sane MUC through a sane server.
  201. lovetox so right now i stay with not adding it and discoing jids if i receive carbons
  202. Ge0rG Holger: </s>
  203. Ge0rG lovetox: you can skip the disco if the incoming message contains an <x/>
  204. Ge0rG lovetox: and obviously if you are already joined
  205. Ge0rG Holger: sorry.
  206. lovetox of course i disco only if i dont know the jid, thats obvious
  207. Ge0rG Holger: there is currently no way for a client to detect this behavior.
  208. Ge0rG Holger: and I'm not sure it would be smart to add a stream feature to the server "I will still CC tagged MUC-PMs"
  209. Holger Ge0rG: I would just optimize your rules for the case where all 2/3 sides adhere to them.
  210. Ge0rG Holger: I think the smart thing for the server would be to keep track of MUCs and CC accordingly
  211. Holger Ge0rG: Plus workarounds for broken peers if and only if they don't hurt.
  212. Ge0rG Holger: I optimize my workarounds for maximum message delivery.
  213. Holger Then servers should always carbon PMs.
  214. Holger Otherwise, Maranda's MUC messages won't be delivered.
  215. Holger MUC PMs.
  216. Ge0rG Holger: just kick Maranda.
  217. Holger Maybe there's other servers who don't fork?
  218. Holger What do I know.
  219. Holger Well, you're the boss. I'll shut up and reload the wiki and follow the rules :-P
  220. Ge0rG Holger: a MUC that depends on a remote entity configuration setting it can't see or know is broken.
  221. Holger You also call ejabberd broken because it follows your rules as of a few months ago, rather than todays. Why add a workaround for this brokenness but not for another?
  222. Ge0rG Holger: the rules haven't changed in the last 6 months, I was just briefly confused today
  223. Holger Unless I'm totally mistaken, the idea that clients should add <x/> and carbon modules should copy those marked outgoing PMs was the most-recent one; and didn't exist last time I touched the code. But whatever, this is not important.
  224. Ge0rG Holger: that idea already existed half a year ago, I'm just not sure over which channel it was communicated
  225. Holger Yes "few months" might be wrong, sigh.
  226. Ge0rG Holger: pardon me my harsh words. You are doing a great job at cleaning up this mess of a protocol.
  227. Holger Who cares. The question is whether that last workaround is worth it, we disagree on that but that's fine.
  228. Holger I still believe that MUC PMs aren't fixable anyway, so I don't care that much. I'd just be a bit unhappy with having to tell client authors to do a disco#info here. I'd expect any sane author to do "WTF?". But on the upside, there's no new XMPP client authors anyway.
  229. Ge0rG Holger: the 100% solution is (b) from - the server needs to know which MUCs each client is joined to and prevent CCs accordingly
  230. Ge0rG Holger: and inject <x/> into the other CCs
  231. Ge0rG Holger: if the server appropriately injects <x/>es, the client never needs to disco#info
  232. Holger Yeah. I'm still trying to avoid tracking MUC presence, but yes this can't be solved 100% without.
  233. Holger And IIRC Zash came up with weird corner cases where tracking goes wrong? MUC service returning a different nick or something? Then again I don't see why the server couldn't also track that.
  234. Holger So yes in the end that's probably the best solution.
  235. Ge0rG Holger: yes, the server needs to track that
  236. Ge0rG Holger: so the server needs to be aware of MUC-pushed nickname changes
  237. Ge0rG the real fun starts when you have two clients joined to a MUC with different nicknames.
  238. Holger Right.
  239. Ge0rG then you must CC only to the nick-sharing clients
  240. Holger Yup.
  241. Holger XMPP is smart servers! \o/
  242. Link Mauve ivucica, is your server already public?
  243. Holger Ge0rG: But if we don't also solve MAM for MUC PMs, the result will still suck.
  244. Ge0rG Holger: right.
  245. Ge0rG Holger: but I don't see an easy way forward right now.
  246. Ge0rG Holger: it's the same problem like with Carbons. They need to be ignored if the client is not joined
  247. Ge0rG but being (not) joined is a temporary status thing
  248. Holger Yup. The temporary status thing sucks. Elsewhere we're trying to drop the presence-requirement for MUC, but for PMs this won't work.
  249. ivucica > ivucica, is your server already public? No, it doesn't even do routing yet. I will also have to go through my employer to release it when the time comes.
  250. ivucica The software is not up and running at this time, either. I'll do some further experimenting myself when I catch time :-)
  251. Link Mauve Which language is it written in? And which features do you plan on putting focus on?
  252. ivucica Go, and I am experimenting with writing a clustered service. Different processes handling c2s, s2s, sessions ("streams"). Router would contact a directory system to find where to route a stanza (unless fulljid / stream is known locally).
  253. ivucica Very experimental, and in the bucket of "I don't know if it makes sense nor whether it will keep my interest". Its streaming processor is not ideal and I don't have a fully clear picture on how to do tag handler registration (probably keep the tree-thus-far as a queue, then match it against an XPath). If a tag is handled locally, handler would be responsible for maintaining the stack and matching any children, probably.
  254. Holger ivucica: Different "processes" as in goroutines, or seperate OS processes?
  255. Holger I guess the latter, because clustered?
  256. ivucica Separate OS processes on different machines
  257. Holger I see.
  258. Holger jabberd2 does something similar (in C).
  259. Holger Though they're trying to get rid of that design ;-)
  260. Holger (<> -- "Merging separate daemons to one.")
  261. ivucica Seems interesting. Especially as they say they want to *allow* for putting all in one binary.
  262. ivucica And that *most* deployments are not distributed.
  263. ivucica I wonder how their router works, as it was quoted as a source of problems. I planned to use 1 or more etcd as backing stores of truth for routing.
  264. ivucica (if I reach that phase :))
  265. ivucica It is out of no personal need, so I am in no rush nor pressure to get results :)
  266. ivucica Multiple etcd deployments with routers using consistent hashing for picking which one to refer to would allow for scaling beyond capacity of one etcd cluster for finding a particular fulljid's resources and streams :)
  267. Holger ivucica: Not sure decoupling session handling from c2s/s2s connections is worth it, by the way. We got rid of that in ejabberd recently as nobody made use of the separation.
  268. ivucica Holger, i get it, but this is personal toy project
  269. Holger ivucica: Yes sure, not trying to stop you from anything :-)
  270. Holger I see how playing with these things is fun.
  271. ivucica i would imagine that at a large enough scale it would be worth it, it's just that almost nobody has a 'large enough scale'
  272. ivucica definitely not me
  273. ivucica :>
  274. Holger Well for ejabberd it was the opposite, it was too expensive for no benefit at large scale.
  275. ivucica I don't know much about ejabberd's internal structure
  276. ivucica Doesn't its clustering behave like a mesh?
  277. ivucica All I know is that I was super impressed by scalability of ejabberd even at what seemed to me like a low number of nodes
  278. Holger Well yes admittedly the amount of the cost was partly due to implementation specifics. But of course you'll always introduce *some* overhead by separation, and I don't see what this one buys you.
  279. Holger Scalability-wise, I mean.
  280. ivucica The way I see it (and I am probably wrong, as I certainly haven't run a realtime chat service at this scale!), it may depend on how localized your service is.
  281. Holger ivucica: Yes it's just using the clustering you get for free from the Erlang VM, which is a full mesh indeed. I.e. doesn't scale up to many nodes.
  282. Maranda It's nice to find a wall of text after you're done killing yourself with weights Holger, hmm where to start Ge0rG aside... 🤣 🤣 Let me think what I do for forking on MUC pms
  283. ivucica You may decrease latency substantially by having localized clusters that nonetheless know how to find all other users within the same domain.
  284. ivucica Imagine a game, and you want a cluster of people in e.g. Vietnam experience super low latency, while also people in US should experience low latency. I don't know how this would work with ejabberd?
  285. ivucica And by that I mean I literally don't know, as I don't know about its clustering infra.
  286. Maranda A) I *try* to assert how/what based on directed presences.
  287. ivucica Anyway, one very, very recent postmortem that I read about scaling to ridiculous number of concurrent users: Epic Games' Fortnite at 3.4 million concurrent players:
  288. Maranda B) If a resource is already the recipient of the message it wont get a cc as well, other merged resources will get a cc if they're flagged to do so
  289. ivucica I somewhat suspect Epic's "Epic XMPP" is a deployment of ejabberd or Mongoose, though they didn't name either.
  290. Holger ivucica: All I'm saying is that I don't see the point of putting the c2s connection handling of a given client onto one node and his session's handling onto another. Maybe I just misunderstood your separation.
  291. Ge0rG Maranda: a - are you tracking MUC initiated nickname changes?
  292. Maranda Holger that sums it up simply without elaboration I suppose I'm not sure why Ge0rG wants to kick me
  293. ivucica Holger, XEP-0198 is the reason
  294. Holger ivucica: And no Erlang/ejabberd doesn't support geo clustering.
  295. ivucica If you split the concerns up and you want to upgrade c2s servers, you can cleanly disconnect users
  296. Maranda Ge0rG I think that is done automatically yes
  297. Holger (Well the Business Edition does actually, that's using protocol on top of Erlang.)
  298. Holger ivucica: Yes Epic is using ejabberd it seems.
  299. ivucica They reconnect to a different c2s frontend, their session resumes
  300. Ge0rG Maranda: b - I was told that your MUC implementation only sends PMs to one resource and relies on carbons for the forking
  301. Holger ivucica: Yes ejabberd supports resumption on another node without splitting c2s/session handling.
  302. ivucica Effectively, users experience nearly no downtimme this way
  303. Holger ivucica: Client connects to another node and that node grabs all session state from the old one.
  304. ivucica *shrug* I can keep it in the same process too, it's nearly no difference.
  305. Holger ivucica: Yes this was just a side note :-)
  306. ivucica Well "grabs all session state from the old one" -- but the old one is gone
  307. ivucica I assume you can keep the state in Mnesia or something?
  308. Maranda Ge0rG that's right only the one "most available" the thing you like much
  309. Holger ivucica: You could, but there's no point for us.
  310. Holger ivucica: The old one is only gone if it crashed.
  311. ivucica *shrug* yeah, you can probably do something like what's done in VM world: live migration
  312. ivucica Is it really? Again, software upgrade.
  313. Holger ivucica: During software upgrade you keep the old node running for some minutes to hold the state.
  314. ivucica I mean, I know Erlang has *something* that helps with live patching processes.
  315. ivucica Ah, so a drain + something akin to a live migration. Got it.
  316. ivucica Yeah, it's certainly an option and you certainly don't need to split c2s<->stream concerns.
  317. Holger Yes apart from all that you can hot-upgrade Erlang code :-) But of course you might want to reboot nodes for kernel upgrades or other reasons.
  318. Ge0rG Maranda: what if the joined doesn't support carbons?
  319. ivucica I just see this as one of those "do one thing and do it well" things. c2s can be super thing, and you can maybe structure stream handler in such a way that you can have a non-XMPP frontend as well hijacking the same session handler, same router, etc.
  320. Holger ivucica: Go ahead then :-)
  321. ivucica Which is actually a good point and I should think about it further before hardcoding "ah, just pass on the stanzas-and-not-nonzas onto the router"
  322. ivucica Heh :)
  323. Maranda Ge0rG, in case of no carbons and same priority weight the one who joined first will get the message iirc
  324. ivucica Either way, once it's split up cleanly, it doesn't matter if it's an RPC or if it's a local method call.
  325. Maranda Does that answer you?
  326. Ge0rG Maranda: and the others won't get anything. That's sad.
  327. Ge0rG Maranda: please fix your code to fork PMs to all joined clients, like with regular groupchat messages
  328. Maranda Nick merges are sad I couldnt describe em better
  329. Maranda It's basically another hack on top of a bigger hack what do you expect
  330. Maranda And so I'll make lovetox unhappy with deduping how pretty
  331. ivucica Holger, speaking of... are you aware of a good coordinated distributed load-tester for XMPP?
  332. Holger For load-testing ejabberd we usually use <> ...
  333. Holger Not aware of others but that doesn't mean anything.
  334. Holger Maranda: ^? :-)
  335. lovetox Nah Maranda you dont make me unhappy, i will dedup either way, i cant trust you server devs :D
  336. Maranda I feel my 📱 will drop ruinously... First or later.. I've its fold and it on my left hand a bottle of water hold in the armpit bag on right shoulder and typing with the right
  337. Maranda 🤣 🤣 🤣
  338. Maranda Ge0rG and I'd have to verify what I said because every muc related code is so cumbersome that I don't remember all of it (and that says much about what kind of marvellous thing muc is already). Although I recall fiddling much with muc/cc code to avoid duplicates
  339. Ge0rG Maranda: I'm interested in your results.
  340. Maranda Well the deduplication work was requested specifically for Jappix to avoid it doing any. And for that it worked wheter it's perfect or won't quirk with some specific old client/muc server combo it's not for me to say.
  341. Maranda And stick a "remote" somwhere in the end of that last statement.
  342. Maranda And beside I'm one who agree that "most complexity should stay server side".. Clients aren't really supposed to do deduplication.
  343. Maranda And going on another topic it's beyond me that MAM doesn't still have a *delete all logs* thing yet.
  344. Maranda remembers all the pointless sync'ing discussions.
  345. Maranda ... Left for ":3" who knows. 🤣
  346. Ge0rG Maranda: MAM is full of fail.
  347. Holger Maranda: But I don't want to delete all messages, just the one I sent yesterday!
  348. Ge0rG Maranda: nevertheless, feel fre to implement the "smart server properly deduplicating" solution, but please let your MUC fork over all PMs to all clients, so that it still works with remote servers.
  349. Holger Next step after fixing this: Damn I deleted that message but my friend says he can still see the dick pic in his browser!
  350. Maranda Holger, that's another option 🤣🤣... But still there should be a "wipe logs red button anyways"
  351. Holger Matt will say use 0136.
  352. Maranda Ge0rG it's not that it doesn't atm tbh 🤔
  353. Maranda Or at least never had issues with Prosody or Ejabberd (or M-Link)
  354. Ge0rG Holger: what's wrong with providing another XEP for storage control? It could integrate offline message suppression as well
  355. Maranda Feel free to prove me wrong though
  356. Maranda Syncing
  357. Maranda That was the reason when I ranted about it
  358. Holger Ge0rG: Nothing wrong at all. I think I'd separate offline stuff though.
  359. Holger Actually I don't quite see the problem with 0013.
  360. Maranda "it breaks sync'ing"
  361. Holger ... for the offline stuff.
  362. Ge0rG > Clients aren't really supposed to do deduplication. Good luck fixing that
  363. Maranda I know
  364. Ge0rG Especially with MAM, MUC and reflections that lack message IDs
  365. Ge0rG Or, as I recently ranted:
  366. Maranda Tbh I wanted to eliminate nick merges support of Metronome alltogether but I was firmly barred in doing so, SouL make that emote please🤣🤣🤣
  367. Ge0rG Maranda: can you please stop flooding that emoji? :P
  368. Maranda If you say please
  369. Maranda ¯\_(ツ)_/¯ there found it
  370. Ge0rG Maranda: thanks.
  371. Maranda (the SouL emote)
  372. Ge0rG Maranda: my client replaces Unicode Emoji with their ascii name representation enclosed in ":", so the SouL emoji is actually shorter than \:rolling_on_the_floor_laughing\:
  373. SouL Sorry, I was getting out of a meeting ¯\_(ツ)_/¯
  374. pep. Ge0rG, haha, still with that plugin
  375. Ge0rG is running an XMPP server on ツ
  376. Ge0rG pep.: sure
  377. Ge0rG pep.: much better than seeing opaque boxes.
  378. Maranda Oh Ge0rG but SouL's emote isnt rofl 😁
  379. Maranda It was meant for that I was barred eliminating nick merges
  380. Ge0rG Maranda: one way or the other, you are emitting a high number of ROTFLs.
  381. Ge0rG Maranda: I'd suggest switching to :face_palm;
  382. pep. Ge0rG, I see emojis more or less correctly :)
  383. Ge0rG Maranda: I'd suggest switching to 🤦
  384. Maranda ☠️👻🤦‍♂️
  385. Maranda There found it but I like SouL's ascii more 🤷‍♂️
  386. Ge0rG Maranda: also for <reasons> your facepalm is explicitly coded as female.
  387. Maranda O.o
  388. Ge0rG As is your shrug emoji.
  389. Maranda is confused, sent male versions.
  390. Ge0rG Oh wait. I'm confused.
  391. Ge0rG Maranda: also for <reasons> your facepalm is explicitly coded as *male*.
  392. Ge0rG Sorry.
  393. pep. Ge0rG, because why would the default be male </feminist> :P
  394. pep. (No, I do not care)
  395. Ge0rG pep.: on my platform, the default face palm is female, but most other defaults are male.
  396. pep. Interesting
  397. debacle Ge0rg, how exactly do you know whether they are male or female? XX vs. XY chromosomes?
  398. ThibG (I think it rather has to do with comparing the default emoji with the explicitly-gendered ones)
  399. m4lvin Someone here who can edit the website? The link to 1.0.0-beta2 from is broken
  400. m4lvin I think it should be ?
  401. lovetox yes, but the file should be renamed to beta1
  402. lovetox * beta2
  403. lovetox Asterix
  404. Asterix fixed
  405. m4lvin 👏
  406. m4lvin another question, is there something like a portable gajim for linux? a package including nbxmpp etc. that I can run without installing anything globally? say, because I don't have sudo...
  407. lovetox we have flatpak builds
  408. m4lvin ahh
  409. lovetox i dont know if that can be run without sudo
  410. mimi89999 What is the situation with the `messagewindow` branch?
  411. bot Michel Le Bihan created an issue in _gajim_ < >: #8908: < High RAM usage / memory leak >
  412. lovetox situation is that i have to fix bugs for gajim 1.0 release, instead of investing time into new features
  413. Ge0rG It would also be great to have a "Join MUC" dialog that doesn't split the MUC JID into two separate things.
  414. marc Yes, same for account creation / IBR
  415. lovetox but then we cannot show recently used servers
  416. marc You can but with a bit more code 😁
  417. lovetox yeah maybe some autocomplete method
  418. lovetox Holger, i cannot dedup this
  419. lovetox if i receive muc pm, and are joined with 2 clients
  420. lovetox and the muc forks the message to each client
  421. lovetox my server will record it as 2 messages incoming, hence 2 different stanza-ids
  422. lovetox lol how broken is this
  423. lovetox Ge0rG,
  424. lovetox also droping received carbons seems like a way to fix live messages on most servers, but on mam catchup i get again the message 2 times and cannot dedup with stanza-id
  425. lovetox this essentially leaves me with trusting the origin-id of the sende IF he uses it
  426. lovetox i think im not going to fix this, people who are joined with 2 clients, just have to live with getting pms doubled
  427. marc Sounds ugly :D
  428. apollo13 I already get duplicated messages when I use conversation and send a message to my gajim instance
  429. apollo13 (same user, two resources)
  430. apollo13 happens only in the conversation -> gajim direction
  431. apollo13 the other direction is fine
  432. apollo13 imo gajim should be able to deduplicate that, here are the stanzas I receive:
  433. apollo13 is that something fixable lovetox?
  434. lovetox didnt you report this already?
  435. apollo13 ah might be
  436. apollo13 I just remember it now since you were talking about it :)
  437. apollo13 lets see if I can find the ticket
  438. apollo13 ah yes
  439. Ge0rG lovetox: you are speaking of MAM, not of Carbons?
  440. lovetox hm yes, Ge0rG if a muc forks messages, then they are put two times into MAM
  441. Ge0rG lovetox: that's two times more than I had expected.
  442. lovetox but then its impossible to dedup
  443. Ge0rG lovetox: it would be great to ask standards@ about how to handle MUC-PM MAM in principle.
  444. lovetox only with time and body
  445. lovetox which is awful
  446. Ge0rG lovetox: because the original message ID is lost in MAM?
  447. lovetox original-id is not something i can depend on
  448. lovetox its only unique inside the senders DB
  449. lovetox not in mine
  450. Ge0rG lovetox: I'm talking of message ID, not of origin-ID
  451. lovetox message-id is not unique?!
  452. lovetox i can you send 100 messages with the same message-id
  453. Ge0rG lovetox: so you are overriding origin-ID on incoming messages?
  454. lovetox on an incoming message, the server injects the stanza-id
  455. Ge0rG lovetox: I understand your principal problem, and I don't know yet how to properly solve it.
  456. lovetox and this is what i save
  457. lovetox yes of course i could save, stanza-id, origin-id, message-id build a hash over it
  458. Ge0rG lovetox: the "real" solution would be to make your account maintain MUC presence for you
  459. lovetox thats just insane but ok :D
  460. lovetox no i cant even do that
  461. lovetox as stanza-id is not the same so hash is not the same
  462. lovetox no its impossible
  463. Ge0rG lovetox: you could use message-id and origin-id
  464. lovetox nah, this is a server issue now for me
  465. Ge0rG lovetox: it surely is
  466. lovetox so beside that i have the same problem now with messages to self
  467. lovetox they are put in 2 times into mam
  468. Ge0rG lovetox: messages to self are another can of worms.
  469. Ge0rG lovetox: and they are CCed to you
  470. Ge0rG lovetox: sometimes you even might get two CCs
  471. Ge0rG lovetox: and some servers send CCs with incorrect from/to fields
  472. Ge0rG I'm planning to write an XEP for them, called Self-Message Scratchapd (SMS)
  473. lovetox hm look at that
  474. lovetox thats a message to myself
  475. lovetox <message id='c30c97c2-8dd2-4b41-b3a1-d258f006eb60' type='chat' from='' to=''> <body>hi</body> <origin-id id='c30c97c2-8dd2-4b41-b3a1-d258f006eb60' xmlns='urn:xmpp:sid:0'/> <thread>utXGrOHxkTdJhZvRYjlWiKguZUBBjzLl</thread> <stanza-id by='' xmlns='urn:xmpp:sid:0' id='2018-02-21-dc919ac78784f1a0'/> <stanza-id by='' xmlns='urn:xmpp:sid:0' id='2018-02-21-877a9b72a095eb60'/> </message>
  476. lovetox its stamped 2 times by my server?
  477. lovetox is this how it helps me to dedup this
  478. Ge0rG lovetox: that's great!
  479. Ge0rG lovetox: another server bug, it seems.
  480. Ge0rG lovetox: ;)
  481. Maranda 🤔
  482. Ge0rG lovetox: so I've written a problem statement to standards@, let the dog fights begin
  483. lovetox thanks
  484. Maranda Release the war hounds!
  485. Maranda 🤣
  486. Ge0rG Maranda: did you check your PMs forking already?
  487. Maranda Not yet
  488. Maranda Hmmm
  489. Maranda Ge0rG, apparently I recalled wrongly, I fork to all merged resources, but mod_message_carbon specifically ignores muc pms asserting directed presence.
  490. Maranda ¯\_(ツ)_/¯
  491. Ge0rG Maranda: what does "asserting directed presence" mean?
  492. Maranda Ge0rG, basically it tries to assert if a directed presence is sent to a muc if so it flags that on the user session, then mod_message_carbons uses that to not process PMs forked from a muc. For some reason I thought I did something more convolute *ducks*
  493. Maranda Ge0rG, basically it tries to assert if a directed presence is sent to a muc if so it flags the muc bare jid storing it in a table on the user session, then mod_message_carbons uses that to not process PMs forked from a muc. For some reason I thought I did something more convolute *ducks*
  494. Ge0rG Maranda: do you catch presence with an <x> element inside?
  495. Maranda yes and muc xmlns
  496. Maranda Rereading, the pipermail message you posted earlier I think at the time I did come up that muc private messages should be properly flagged with a <x> element but since that's never done and probably never will in the foreseable future I revolved to tracking directed presences.
  497. Maranda Ge0rG,
  498. Maranda But considering we're talking about 3 years ago or so ¯\_(ツ)_/¯
  499. Maranda I even managed to forget what I exactly did :P
  500. Ge0rG Maranda: 👍
  501. Maranda (or rather almost 4 XD)
  502. Ge0rG But my email is from January 2017?
  503. Maranda Ge0rG, well I had to deal with it *way* before that ;)
  504. Ge0rG Maranda: where is your email to standards?
  505. Maranda Ge0rG,
  506. Maranda points date.
  507. Maranda Ge0rG, I wasn't joined at the time I think?
  508. Ge0rG Maranda: that's no valid excuse
  509. Ge0rG Maranda: will you still cc PMs sent by your local user?
  510. Maranda ¯\_(ツ)_/¯ I did discuss that issue with waqas if memory serves right though
  511. Maranda Ge0rG, what do you mean?
  512. Maranda My own pms or what else?
  513. Ge0rG Maranda: I mean generating "sent" carbons
  514. Maranda I do
  515. Ge0rG Great
  516. Maranda (but they follow some more rules to avoid duping for the one sending it at least)
  517. Maranda (still)
  518. Ge0rG Maranda: it's nice to see that people have thought about the problem already. But it's sad this knowledge never got public
  519. Holger lovetox: I think Conversations does more dedup magic such as trusting the message ID if it looks like a UUID or whatever. But yes this is horrible.
  520. lovetox i use now origin-id
  521. lovetox this is in no way perfect, but whatever, as i know the devs of aprox 80% of all xmpp clients out there
  522. lovetox i trust they use a uuid
  523. bot Philipp Hörist modified an issue in _gajim_ < >: #8812: < Sending to gajim from another resource with carbon copies enabled duplicates the message. >
  524. bot Philipp Hörist modified an issue in _gajim_ < >: #8907: < Conserve the aspect ratio during avatar resizing >
  525. bot Philipp Hörist modified an issue in _gajim_ < >: #8901: < Acconut name isn't change in the popup menu of notification area icon >
  526. bot Philipp Hörist modified an issue in _gajim_ < >: #8509: < Unicode Emojis broken on Windows >
  527. bot Philipp Hörist modified an issue in _gajim_ < >: #8899: < gajim crashes when moving a window by dragging >
  528. bot Philipp Hörist modified an issue in _gajim_ < >: #8894: < Can't treat XMPP URI correctly >
  529. bot Philipp Hörist modified an issue in _gajim_ < >: #8902: < Gajim can not open url from vcard... Gajim past mailto to url button. >
  530. bot Philipp Hörist modified an issue in _gajim_ < >: #8895: < Can't open chat >
  531. bot Philipp Hörist closed an issue in _gajim_ < >: #8812: < Sending to gajim from another resource with carbon copies enabled duplicates the message. >
  532. bot Philipp Hörist pushed 3 commits to branch _refs/heads/master_ of _gajim_ < >:
  533. bot Philipp Hörist modified an issue in _gajim_ < >: #8904: < Wrong checksum for in org.gajim.Gajim.json for Flatpak install >
  534. lovetox im probably foolish but i hope all duplicate cases are now solved
  535. debacle lovetox, how about not deduplicating, but duplicating yourself? this would help Gajim a lot! :~)
  536. lovetox 🤣
  537. bot Philipp Hörist pushed 1 commit to branch _refs/heads/master_ of _gajim_ < >: *70e0bcc5* < > Pass jid as string to find_stanza_id()
  538. bot Philipp Hörist pushed 1 commit to branch _refs/heads/master_ of _gajim_ < >: *1fbc6a23* < > Tag MUC PMs This gives the server the chance to react accordingly without applying much logic. Also it makes it easier for us to recognize MUC PMs in MAM querys
  539. Maranda lovetox, nice too bad you're the only one 🤣
  540. Maranda Rather Gajim is
  541. bot Philipp Hörist pushed 1 commit to branch _refs/heads/master_ of _gajim_ < >: *eb3a53c7* < > Refactor emoticon theme combobox - Use ComboBoxText, its much simpler - Add a dedicated method that returns all available themes - If the configured Theme is not available fallback to font-emoticons
  542. bot Joachim Opdenakker created an issue in _gajim_ < >: #8909: < Gajim-git resets locale >
  543. bot Philipp Hörist pushed 1 commit to branch _refs/heads/master_ of _gajim_ < >: *52fa5779* < > Better emoticon theme fallback strategy Fallback must be happening in init_emoticon() instead of PreferencesWindow
  544. Holger Ge0rG: FWIW, ...
  545. bot Joachim Opdenakker closed an issue in _gajim_ < >: #8909: < Gajim-git resets locale >