Gajim - 2018-02-12

  1. rom1dep ok, for the record, I can get gajim to crash when starting a muc PM ```python3: cairo-surface.c:930: cairo_surface_reference: Assertion `CAIRO_REFERENCE_COUNT_HAS_REFERENCE (&surface->ref_count)' failed.``` something avatars-related?
  2. lovetox hm maybe
  3. lovetox on every muc pm rom1dep ?
  4. lovetox test with contact where you dont have an avatar
  5. lovetox i cant reproduce this here
  6. rom1dep lovetox: I'm out to work, I'll test that this evening
  7. rom1dep The contact did have an avatar
  8. bot Philipp Hörist pushed 3 commits to branch _refs/heads/master_ of _gajim_ < >:
  9. bot Philipp Hörist merged a merge request for _gajim/master_ < >: Flatpak: bundle plugin installer
  10. bot Philipp Hörist pushed 4 commits to branch _refs/heads/gajim_1.0_ of _gajim_ < >:
  11. bot Philipp Hörist merged a merge request for _gajim/gajim_1.0_ < >: Flatpak: bundle plugin installer and use stable branch
  12. bronko lovetox, latest windows nightly is working again, thank you for fixing it
  13. bot Marcin created an issue in _gajim_ < >: #8889: < Messages received via MAM are not shown in the current chat window >
  14. bot Marcin created an issue in _gajim-plugins_ < >: #268: < [OMEMO/HTTPUpload] Cannot open HTTPUpload-ed AES-GCM encrypted photos from the chat history >
  15. bot Philipp Hörist closed an issue in _gajim_ < >: #8036: < Support for XEP-0313 in MUCs >
  16. bot Philipp Hörist closed an issue in _gajim-plugins_ < >: #268: < [OMEMO/HTTPUpload] Cannot open HTTPUpload-ed AES-GCM encrypted photos from the chat history >
  17. bot Philipp Hörist modified an issue in _gajim_ < >: #8036: < Support for XEP-0313 in MUCs >
  18. marmistrz lovetox, btw. did you consider using type annotations and mypy linting in the project?
  19. bot Philipp Hörist closed an issue in _gajim_ < >: #8889: < Messages received via MAM are not shown in the current chat window >
  20. marmistrz it could make it easier to avoid the "None has no such method" or similar errors
  21. lovetox we use pylint, but it cant detected those things reliably
  22. lovetox i heard about type annotations but didnt look into it
  23. lovetox is it not a python3.6 feature?
  24. Link Mauve 3.5.
  25. Link Mauve But it was already available in 3.4.
  26. lovetox btw we run pylint on every commit, so it would show the "thing dont has this method"
  27. lovetox but as i said, pylint does not detect this in a lot of cases
  28. Link Mauve Thanks dynamic programming…
  29. Link Mauve lovetox, mypy is a workaround where the code explicitly describes which types can be taken by a variable.
  30. lovetox so it build on the annotation feature
  31. Link Mauve Yeah.
  32. lovetox nice, so if it only checks those things where we give annotation, then i think this can be useful
  33. rom1dep type annotation on the function itself (w/o which the whole thing is pretty much useless) came later, like with 3.5 or 3.6. That may be still to recent for most LTS distros
  34. lovetox but i think i read somewhere that they are backwards compatible
  35. lovetox they may not work in prev python versions
  36. lovetox but having them written in code, does not lead to an error
  37. Maranda lovetox so can I update win master now 🤗?
  38. lovetox yes
  39. Maranda Ok 👍
  40. Maranda lovetox decided yet on the muc favicons by the way?
  41. lovetox yes i would implement it, but dont know when
  42. Maranda 😁
  43. SaltyBones I've tried type annotations in python and found them unconvincing. Having some tests is much better.
  44. mathieui one is not supposed to replace the other
  45. SaltyBones But in practice both require making an effort... :)
  46. SaltyBones I'm not saying don't use type annotations; they can be very useful in some cases. However, in most cases giving the wrong thing to a function fails pretty obviously anyway and you don't have to maintain the annotations.
  47. lovetox this probably doesnt work for named arguments where i define a default value
  48. lovetox like def func(a=None) but a is normally a str
  49. SaltyBones you can do unions...
  50. SaltyBones a: Union[str, none]
  51. SaltyBones something like that
  52. SaltyBones I'm not an expert. I tried once and didn't notice any benefits so I stopped again. :)
  53. Link Mauve SaltyBones, Optional[str]
  54. SaltyBones Link Mauve, I guess you are actually using type hinting? :)
  55. Link Mauve Not much.
  56. Link Mauve Only for new projects, and I will never again start a big project in Python.
  57. mathieui adding type hinting to already-existing projects is a real pain, too
  58. mathieui but somehow ale.vim has type hints checking builtin, which makes it much better
  59. Wiktor Hello, I've seen gajim 1.0.0-beta1 in several repositories (AUR, but on the webpage there is no mention of it, also here: is this something that is slowly being rolled out everywhere? Thanks!
  60. SaltyBones Wiktor, the publicity team is working hard on updating the website. Unfortunately it doesn't exist so progress is slow. xD
  61. lovetox yes we raised the version but didnt have time until now to update the site and rename a package to beta
  62. lovetox the debian maintainer was faster :)
  63. lovetox but it could happen today ^^
  64. Wiktor I just wasn't sure if I missed something, thanks a lot for your hard work! Gajim really looks and feels very, very nice.
  65. Wiktor SaltyBones: I'm not into pretty websites, it's the "latest version binary" that makes me truly happy ;)
  66. Wiktor by the way, is there some kind of way to get notified about new releases? RSS or something like that?
  67. debacle Wiktor, in the XMPP world, one would use PubSub for that :~)
  68. Wiktor Is there a PubSub node then? :)
  69. debacle Not that I'm aware of. asterix, lovetox, how about a pubsub news channel for Gajim? Or maybe one for XMPP clients in general? (People interested in Gajim are probably interested in other clients, too.)
  70. Wiktor Last time I checked only Gajim could receive Atom messages from a PubSub node, and even then only the subject was rendered (no body).
  71. Wiktor debacle: I think the easier solution would be just a MUC room only for new version release announcements.
  72. Wiktor still XMPP but easy to manage.
  73. debacle Wiktor, client support is much better for MUC than for PubSub, yes.
  74. debacle Not only Gajim, most clients are not PubSub aware.
  75. Wiktor Yes, sadly, I was also dreaming about a world where most clients would implement most XEPs but it turns out lots of XEPs is a lot of work and added complexity :)
  76. Link Mauve Ah, I still get this traceback on master! Traceback (most recent call last): File "/home/linkmauve/packages/gajim-git/src/gajim/gajim/", line 5627, in query_tooltip tooltip.populate(connected_contacts, account, typ) File "/home/linkmauve/packages/gajim-git/src/gajim/gajim/", line 413, in populate self.contact_jid = self.prim_contact.jid AttributeError: 'NoneType' object has no attribute 'jid'
  77. debacle I think, Gajim could take the lead there (after 1.0.0, of course) by having tabs with PubSub feeds (one tab per feed) and render Atom to plain text.
  78. Wiktor debacle: are you volunteering for implementation? :)
  79. debacle Well, I would at least look into the matter. I did not do any work on Gajim or nbxmpp, but I'm Pythonite and start to get some knowledge of XMPP. I implemented basic PubSub support for my little bot "painintheapt" yesterday, using sleekxmpp.
  80. debacle
  81. Wiktor Oh, very cool, I'm was also thinking about something like that (before I figured out no client could render Atom/PubSub properly)
  82. Wiktor personally I would rather want Gajim to be slimmed down (removing features or moving them to plugins), then it's easier to work on any potential bugs
  83. SaltyBones debacle, that is indeed pretty cool! :)
  84. debacle Wiktor, I like plugins very much and would like to see more functions in plugins rather in the core, too. But this would require many more (stable!) APIs inside of Gajim than currently are available. Developers are already busy and cannot easily make more APIs. Very often, it is less work to move something into the core, unfortunately. E.g. the HTTP upload plugin has just been cancelled and moved to Gajim, because it allows things a plugin cannot do without a lot of API work.
  85. debacle As for PubSub: I use and there it is easy to create PubSub nodes, publish to them, subscribe etc. It would be even more cool (freezing!) if I could follow a node using Gajim.
  86. Wiktor debacle: well HTTP upload is not that complex, besides it's kind of becoming "standard" solution for seamless file/image sharing
  87. Wiktor Yes, I've seen movim nodes, they have their "news" in PubSub (besides RSS), that's cool :)
  88. debacle "kick ass" is their markting slogan :~)
  89. nico this apt bot is really nice 😀 I would like my teamspeak managment bot to be like that 😀
  90. SaltyBones debacle, really? Which functionality does http_upload have now that didn't work in the plugin?
  91. debacle SaltyBones : ask lovetox or asterix, I don't remember.
  92. SaltyBones lovetox, you have been summoned!
  93. bronko SaltyBones, drag&drop support
  94. SaltyBones
  95. SaltyBones oh yeah, that works
  96. debacle I love this picture.
  97. stp debacle: Regarding Movim: do you log in there with your existing XMPP-account or do you create an extra one?
  98. mathieui you can do both
  99. Link Mauve stp, it’s a normal XMPP client, the main difference is whether you care about giving your credentials to the admins of the Movim instance you want to use or not.
  100. Asterix Wiktor: there is a announce ML
  101. Wiktor Asterix: great!
  102. Wiktor for reference:
  103. Wiktor by the way someone mentioned drag&drop and http upload, for me drag&drop currenly uses direct (jingle?) file transport, is there a way to change it to always use http upload?
  104. debacle I created my own account, because I'm not willing to give my "main" XMPP credentials to a web application :~)
  105. Wiktor but, but, but... it's gajim's web application!
  106. bot Link Mauve proposed a new merge request for _gajim/master_ < >: Fix data form window space usage
  107. SaltyBones another thing that could be decided on the differentiation of pubchat vs privgroup: Do you create previews for all images and links.
  108. lovetox Wiktor, are you sure your server supports httpupload?
  109. Wiktor Yes, because I've got HTTP Upload button here and it works
  110. lovetox because httpupload is the preference already
  111. Wiktor (I'm running alpha2 though)
  112. lovetox if you have a httpupload button then you use a older gajim version
  113. lovetox which has no drag&drop
  114. lovetox no drag&drop for httpupload
  115. lovetox it only works with beta1
  116. Wiktor ah, okay, great
  117. Wiktor wait, no button in beta1? so only drag&drop?
  118. lovetox here is a button, but not from the plugin anymore
  119. Wiktor ok ok
  120. lovetox there is a general file transfer button
  121. lovetox that uses either your pref or if its not available the transfer that is available
  122. Wiktor by the way, if I'm not mistaken, before 1.0.0 the image file chooser dialog had preview, and now the preview is gone (it looks like generic file open dialog), is that a known issue?
  123. Wiktor okay, that sounds smart
  124. lovetox yes its known
  125. Wiktor ok!
  126. lovetox though im not sure its still the case with beta1
  127. Wiktor it's moving fast :)
  128. Wiktor how about paste from clipboard (CTRL+V) when clipboard contains image? is that something that can be accessed from Python?
  129. lovetox yes there is a open issue about it
  130. Wiktor ok, thank you
  131. bot Philipp Hörist modified an issue in _gajim_ < >: #8875: < kwallet support missing >
  132. Asterix lovetox: can be closed, no?
  133. rom1dep Are the beta-1 builds available for Windows? This morning's download page didn't see to list them
  134. lovetox i clicked the close button 1 minute ago
  135. rom1dep seem*
  136. bot Philipp Hörist closed an issue in _gajim_ < >: #8875: < kwallet support missing >
  137. lovetox Asterix could you save todays windows build as the beta-1 build
  138. Asterix Yes ... I have to ... but I don't have time .... will try ASAP
  139. rom1dep 👍
  140. stp Ok, I'll try Movim with an extra account. I don't get it's principle yet.
  141. bot Philipp Hörist closed an issue in _gajim_ < >: #8593: < UTF-16 surrogate pairs stripped by invalid_XML_chars regex on "narrow" python builds >
  142. Asterix gitlab upgrading
  143. lovetox ah
  144. Asterix done
  145. bot Philipp Hörist closed an issue in _gajim_ < >: #8572: < Windows TLS handshake error >
  146. lovetox its soooo slow :/
  147. Asterix yes fitlab takes 100% CPU ....
  148. Asterix nothing special except that
  149. lovetox i will google some performance settings
  150. rom1dep gitlab being slow? common…
  151. lovetox maybe it does something the whole time we dont need anyway
  152. Asterix why only since a few days
  153. Asterix disk io is ok, load is ok
  154. lovetox maybe its the runners
  155. lovetox the more contributors the more i activated runners for there branches
  156. lovetox because otherwise the MRs are not tested
  157. lovetox *their
  158. lovetox how das RAM utilization look like?
  159. lovetox ah i see in the admin interface
  160. Asterix runners are not on the same VM
  161. lovetox not at the limit
  162. Asterix RAM is ok too
  163. Asterix
  164. Asterix bosh seems to disconnect very ogten :/
  165. Asterix bosh seems to disconnect very often :/
  166. Asterix let me restart prosody, it takes quite lots of RMA
  167. Asterix RAM
  168. Asterix it seems a bot is getting all gitlab pages :/
  169. Asterix yandex bot
  170. Asterix x.y.z.t - - [12/Feb/2018:14:33:00 +0100] "GET /flobee/gajim/blob/4f30b67d9eaaa2c37f09d00d82791fc60d4b3ce1/src/ HTTP/1.1" 200 8644 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +"
  171. Asterix if it takes all files of all commits ... that will be long !!!
  172. rom1dep gajim isn't reconnecting automatically :/
  173. Asterix to server, yes it does. To room, no it does not
  174. lovetox rom1dep, depends on the scenario
  175. lovetox windows, linux for example
  176. rom1dep your server rebooted, and I ended-up in a stalled room
  177. lovetox ah oh that
  178. Asterix that is a prosody bug IMHO. when server is closing, it should send an unavailable presence in all room occupants IMHO ...
  179. lovetox yeah i have to test that, normally the server should send that the room is offline
  180. rom1dep it logs that I have been removed from the room (system shutdown), then 35s after I get a connection refused
  181. Maranda It does afair
  182. Maranda It's just that with libevent the stanzas could not make it to remote entities
  183. Maranda I got it for example
  184. lovetox but indeed in case of something like that gajim does not try every X seconds to reconnect to a closed muc
  185. Maranda I often get a password prompt in rooms when I restart more than the other problem
  186. Maranda With Gajim
  187. lovetox yeah but you restart your own server
  188. lovetox so gajim goes offline too
  189. lovetox in that case it will reconnect to bookmarked rooms
  190. lovetox but if you are just kicked with unavail from some room
  191. lovetox gajim does not try constantly to join it agin
  192. lovetox and i dont know if it really should
  193. Maranda Nope, you dont know if and when a server will come back online
  194. Maranda Not from a protocol perspective iirc, so it's correct it doesn't keep to try on system-shutdown
  195. bot Yann Leboulanger pushed 1 commit to branch _refs/heads/master_ of _website_ < >: *d4a52095* < > update website for beta1 release
  196. bot Philipp Hörist merged a merge request for _gajim/master_ < >: Fix data form window space usage
  197. bot Philipp Hörist pushed 1 commit to branch _refs/heads/master_ of _gajim_ < >: *c02af8f5* < > Fix data form window space usage
  198. Wiktor Phew, I just updated to beta1 on Windows and it did *not* go smoothly... by the way what's the recommended way of updating portable installations? Because I just remove everything except UserData and install another portable version. It seems Gajim got stuck migrating Config file printed some kind-of related text (can paste here) and shut down.
  199. lovetox yeah thats basically the only way i think
  200. lovetox delete everythin except UserData
  201. lovetox this could be probably done by the installer
  202. lovetox please post the error regarding the config file
  203. Wiktor
  204. Wiktor and that's all, after short blink of something that looked like a window it shuts down
  205. Wiktor I think I cheated by using version = 0.99.1 in old config file (I think that stopped migration from working)
  206. Wiktor Gajim forgot the account password but that's not important as it's easy to fix
  207. Wiktor Funny thing that it printed some German (Geben Sie den Wert ein) while I don't use German locale, fonts or anything :)
  208. lovetox Asterix MUC Log is broken
  209. lovetox Wiktor can you post the pastebin url
  210. lovetox converse.js does not show it
  211. Wiktor sure,
  212. lovetox oh that thing, i get it from time to time, it has nothing to do with migration i think
  213. lovetox that you changed the version in the config file could lead to multiple problems
  214. Wiktor well, it left the dot config file (.Config), usually it disappears
  215. lovetox as we do database migration based on the version
  216. lovetox but i guess if you didnt see errors until now
  217. lovetox you are fine
  218. Wiktor hmm, maybe I should browse commits what changed between alpha2 and beta1 in migrations :)
  219. lovetox ah ok
  220. lovetox no nothing
  221. lovetox i thought you are upgrading from 0.16.9
  222. lovetox ok but i think i know where the problem is with that bug
  223. lovetox it seems the way we try to delete the .config file does not work on all windows systems
  224. lovetox i have to revisit that
  225. lovetox where did you have the installation?
  226. lovetox maybe on some restricted place?
  227. Wiktor nope, I've got a separate folder that programs can freely modify (*not* program files)
  228. lovetox hm k
  229. Wiktor one way or another I've got backup of that old installation (alpha2), so if there is any potential fix I can try it
  230. Wiktor I think it's dependent on the config file as without config file it works ok (it creates a new default one)
  231. Wiktor but with the old config file it breaks
  232. Wiktor unfortunately I've got 10000 customizations there so it's hard to pinpoint what's the cause
  233. lovetox yeah thats good keep that old backup
  234. lovetox so we can test if i have a fix
  235. Wiktor hmm... btw, I've still got colored OMEMO icon next to message box, I've seen screenshots that have it just black, could that be caused by my franken-migration?
  236. Wiktor (no plugin updates are available)
  237. lovetox we didnt change anything from alpha to beta in that regards
  238. lovetox maybe you can do a screenshot
  239. Wiktor sure, let's try this D&D here
  240. Wiktor
  241. Wiktor
  242. Wiktor the b&w above was found on the internet
  243. Wiktor the color one is what I see
  244. lovetox thats ok Wiktor
  245. lovetox thats not per-se an omemo icon
  246. lovetox it shows up with all encryptions
  247. lovetox if you click on it you get fingerprints and other informations
  248. Wiktor I mean the shield icon not the lock
  249. lovetox this could probably integrated as a menu entry into the lock symbol
  250. Wiktor It looks different
  251. lovetox i mean the shield too
  252. Daniel the bot in this room doesn't cover the website repo, does it?
  253. Wiktor lovetox: ok thanks for info
  254. Asterix [15:28:40] ‎bot‎: Yann Leboulanger pushed 1 commit to branch _refs/heads/master_ of _website_
  255. Asterix Daniel, ^
  256. rom1dep sqlite performance on windows is only sadness…
  257. Daniel Asterix, yes but I just opened a MR and nothing is showing in this room
  258. Asterix ha, for MR, yes probably
  259. Daniel ah i thought it behaves the same for all repos :)
  260. Asterix yep I Saw then while translationg :)
  261. Asterix thanks
  262. bot Yann Leboulanger pushed 2 commits to branch _refs/heads/master_ of _website_ < >:
  263. Asterix it's a gitlab config
  264. Asterix and it's per-project
  265. Daniel I see :)
  266. Daniel ah well, upcoming translations, good point!
  267. Asterix hmm I don't see the result of your MR in :/
  268. Asterix ha conflict in fr and de translations
  269. Maranda SQL Compact?
  270. Maranda :O
  271. Asterix Daniel, you're translating?
  272. Asterix sorry I overwrite what you did because of the conflict
  273. Daniel Asterix, yep, but no problem ;)
  274. Asterix website updated
  275. Daniel on the website it says "TLS, GPG and End-To-End encryption support (with SSL legacy support)". what is meant with SSL legacy support?
  276. lovetox direct TLS
  277. lovetox once it was deemed legacy
  278. Asterix this part (feature list) have to be re-written with 1.0 release
  279. lovetox now everyone uses it again
  280. Daniel thank you both
  281. Daniel other thing I noticed while fiddling with a chat bot.. sometimes bot answers appear before the question is rendered in conversation view. I added some delay to the bot so the answers appear after the question in conversation view. is there a way to solve this?
  282. Asterix XML logs would help to see if it's a client or server issue
  283. Daniel
  284. Asterix indeed server send us the bot reply before our own message
  285. Asterix ha no
  286. Asterix you mean Test is printed before test?
  287. Daniel right, sorry for bad example texts
  288. Asterix then yes, there is something. a priority in our 2 events probably ...
  289. Daniel message order is correct when writing via Conversations, so maybe it is. it's a shame to delay bot answers ;)
  290. Asterix what about history window?
  291. Daniel same in history window
  292. lovetox hm can i talk to the bot?
  293. Daniel hm how do I have to set this up? maybe we can do this via PM?
  294. Daniel it's a simple answer bot realised in sleekxmpp
  295. Daniel typically a delay of 0.6 s is enough for gajim to display the answer after the question
  296. Wiktor I wonder if Conversations uses the delay element to sort them properly (that'd require looking at the xml console)
  297. Daniel Conversations sorts them correctly without an additional delay set
  298. Wiktor How? Maybe it adds own message before server replies
  299. Daniel my example is basically the "boilerplate" described here
  300. lovetox wiktor just because of simple database entry
  301. lovetox if i send a message first it lands first in the database
  302. Wiktor But Gajim doesn't do that currently now?
  303. lovetox it sorts it by time
  304. lovetox but maybe the resolution is not good enough
  305. lovetox but even if not it should be ok
  306. lovetox so there is probably a sort algo problem
  307. Asterix The messageSent event is treated after the incomingMsgEvent it seems. Output of gajim -v ?
  308. concerto Why doesn't Gajim at the very least automatically download all files that the user has sent from another device? Or, even better, automatically download all files sent by someone in the roster?
  309. lovetox because people dont want that
  310. lovetox also we do this already for pictures
  311. lovetox i mean thats kind of horrible, download everything someone sends me
  312. lovetox you really want that Oo
  313. lovetox Asterix i highly doubt that
  314. concerto lovetox: I think it's the commonly expected, desirable behaviour.
  315. lovetox i dont think so sorry
  316. lovetox maybe for really small files
  317. Maranda didn't bot support commands like .xping etc?
  318. concerto lovetox: It's supremely annoying to log into Gajim and realize that your chat history is incomplete (if the discussion was about a file and the file is missing because it wasn't downloaded...) because you hadn't the time to individually click every link in every chat.
  319. lovetox if the message is missing then there can also be no autodownload
  320. lovetox if the message is there
  321. lovetox you can click it
  322. lovetox Daniel
  323. lovetox can you start gajim with gajim.c.logger=DEBUG
  324. lovetox send a message
  325. lovetox and post me the logged db entrys
  326. lovetox or log with -v
  327. lovetox but post only from the moment on you send the message
  328. lovetox then we see also event
  329. bot Philipp Hörist pushed 1 commit to branch _refs/heads/gajim_1.0_ of _gajim_ < >: *172e7af4* < > Read config files with UTF-8
  330. Daniel lovetox, just a moment
  331. Daniel lovetox,
  332. lovetox hm lol
  333. lovetox seems outgoing is recorded with miliseconds
  334. lovetox incoming is rounded down to seconds
  335. Daniel is this a server-side issue or does gajim handle this?
  336. lovetox no gajim issue
  337. lovetox wait i look how hard this is to fix
  338. concerto lovetox: The message is not missing, IIUC. I leave Gajim running on my laptop, and come back to see ten links in each chat which I need to click to maintain a complete history.
  339. lovetox why ? where does the history go?
  340. concerto It's not incomplete in that there are missing messages, it's incomplete because the files aren't downloaded (unless I click on them).
  341. lovetox ok so autodownload for small files
  342. lovetox with current ui this would mean you start gajim and there pop mutliple download windows up
  343. lovetox not very pretty
  344. concerto Ah, yeah, I forgot :\
  345. lovetox once we have the progress inside the chatwindow, i guess this would be an option
  346. bot donpatrice created an issue in _gajim_ < >: #8890: < Unable to insert emoticons >
  347. SaltyBones 🔲‍⛓️
  348. SaltyBones
  349. SaltyBones 🔲‍⛓️
  350. concerto lovetox: can anything be done for the background in message correction with dark theme? I think I had earlier mentioned that it becomes nearly impossible to read...
  351. lovetox yes start with making a issue
  352. lovetox please
  353. concerto Okay
  354. bot Philipp Hörist pushed 1 commit to branch _refs/heads/gajim_1.0_ of _gajim_ < >: *19689ac9* < > Dont use getTimestamp() getTimestamp() generates always a timestring with a resolution of seconds and this means time gets always rounded down. Because we use a resolution of .00000 precision for outgoing messages this can lead to wrongly sorted messages.
  355. bot donpatrice closed an issue in _gajim_ < >: #8890: < Unable to insert emoticons >
  356. bot Philipp Hörist pushed 1 commit to branch _refs/heads/master_ of _gajim_ < >: *fb4b2624* < > Dont use getTimestamp() getTimestamp() generates always a timestring with a resolution of seconds and this means time gets always rounded down. Because we use a resolution of .00000 precision for outgoing messages this can lead to wrongly sorted messages.
  357. lovetox Daniel, last commit shoud solve that
  358. lovetox Link Mauve, the longer i think about message sorting with timestamps
  359. lovetox i feel there are race conditions
  360. lovetox where its impossible to get it right
  361. lovetox mainly because of MAM, because it now allows for holes in your message history, that you want much later to be loaded
  362. lovetox in this case it could happen on the edge of the hole, that you have a message with a timestamp that is exactly the same as the last timestamp you get from MAM to close the hole
  363. lovetox and in this case you cant sort after row id
  364. lovetox these events are very unlikely to happen, but still
  365. Maranda <delay />?
  366. lovetox yes it only has second resolution
  367. lovetox so many messages could have the same timestamp
  368. Link Mauve lovetox, yeah, that’s why you should rely on stanza-id instead of timestamps.
  369. Maranda ^
  370. lovetox seems complicated
  371. Maranda Those aren't supposed to clash, although UUIDs *could* have collisions.
  372. Link Mauve lovetox, uh, <delay/> has microsecond resolution.
  373. lovetox in theory
  374. lovetox though im not aware of any MAM impl that uses it
  375. Maranda lovetox, use stanza-ids
  376. lovetox you can sort UUIDs?
  377. lovetox i doubt that
  378. Maranda or rather uuids + delay
  379. lovetox how is a UUID helping you with order
  380. lovetox its random
  381. Maranda why aren't you just storing on incoming order?
  382. lovetox did you read my problem above?
  383. Maranda lovetox, I'm not sure I understand what the sorting problem is?
  384. Maranda Yes I do
  385. Maranda Yes I did
  386. lovetox then you see that MAM allows it to query various timeframes
  387. lovetox incoming order makes not much sense
  388. lovetox usually you do, timestamp sorting, and if you have messages with the same timestamp you sort again for incoming order
  389. lovetox but this leaves edge cases where it goes wrong
  390. Maranda That's why you should store the messages in receive order (simple key array?), I never had a problem with RSM.
  391. Maranda At least as server, so I can be sure index 1 is the oldest message in the archive etc.
  392. lovetox of course for a server there counts incoming order
  393. Maranda lovetox, I mean you either trust the order of the server or you do your own thing.
  394. lovetox you cant get message in another way then waiting for them to come in
  395. SaltyBones I am amused by the fact that the chat window divider has a tooltip saying "text below this line is what has been said since the last time you paid attention to this group chat"...
  396. lovetox but a client can query any timeframe that he wants
  397. Maranda That's the other option
  398. lovetox if i come online and write a message on day 5
  399. lovetox then one hour later i query day 4
  400. lovetox the incoming order of message in the db
  401. lovetox is clearly wrong
  402. lovetox do you agree ?
  403. Maranda hmm
  404. lovetox yeah welcome in client dev land
  405. lovetox now you need the timestamp
  406. Maranda Not sure, you query just day four or everything from four upto day 5?
  407. SaltyBones lovetox, why would you query day four instead of "everything new"?
  408. lovetox does not matter both scenarios can happen
  409. lovetox a user could just want day 4
  410. lovetox or everything up to day 5
  411. lovetox thing is MAM allows you to query any timeframe at any time
  412. Maranda couldn't you just fill the holes when you need to and trust / match with the server order then?
  413. lovetox SaltyBones, MAM never gives you new messages
  414. lovetox its an Archive
  415. Maranda I mean you write message on 5, then query for four, there's a hole between 4 and 5, should you need to, you query for what's in between and fill the hole relying on server order?
  416. Maranda The timeline *shouldn't change*
  417. lovetox yes but how do save that order
  418. lovetox when i close the application
  419. Maranda *how to save that order*?
  420. Maranda lost the thread..
  421. SaltyBones probably a problem with the timeline
  422. Maranda If you sync with the server mam archive there shouldn't be
  423. Maranda if there're message with the same timestamp you just entrust the order which the server sent you, that's what I mean
  424. lovetox ok i query day 4 gives me as first message (oldest) a message with timestamp = 5
  425. lovetox now i query day 3 gives me as last message also a message with timestamp 5
  426. lovetox hm this is maybe confusing to use days
  427. lovetox but day 4 is later
  428. SaltyBones your problem that you are trying to sync the client and the server view using random chunks?
  429. Maranda you have yymmdd in the stamp though..?
  430. SaltyBones or that the time stamps are lying?
  431. Maranda You'd know what's in 4...
  432. Maranda what's in 3...
  433. Maranda etc.
  434. Maranda or are you talking about the issue with timezones?
  435. lovetox yes that example would not work as the server would always return message with the same timestamp in the same query
  436. lovetox yes SaltyBones
  437. Maranda (beside that times are supposed to be zulu)
  438. SaltyBones lovetox, does that mean both? because I asked an "or" question
  439. lovetox there is no issue with timezone
  440. lovetox random chunks
  441. lovetox if the archive was the only source of messages
  442. SaltyBones so you have random chunks and they have timestamp or they do not have timestamps?
  443. lovetox i would agree there is no problem
  444. lovetox SaltyBones, everything has a timestamp
  445. Maranda You'd assert dates internally in that case, not just the hh:mm:ss
  446. SaltyBones Hm...maybe I need to figure this out in a different way
  447. Maranda and then rely on server order
  448. SaltyBones What is wrong with the following approach:
  449. SaltyBones For every message you get, check by uuid if you have it, otherwise insert it in the appropriate place?
  450. Maranda I'm still failing to see how a message arriving at 22:06:57 on two different days could cause so many issues on randomly queried chunks.
  451. lovetox hm i will construct an example have to think about it
  452. SaltyBones Maranda, I am guessing it has to do with code that is already there and confusing the issue...
  453. Maranda Then fix that code
  454. SaltyBones Maranda, but it requires realizing that there is an issue first and I think we just completed that step ;)
  455. Maranda 🤣 , that's maranda's dumb thinking
  456. Maranda (tm)
  457. SaltyBones
  458. SaltyBones scowls at that emoji. That looks much better as a regular unicode character. :(
  459. Maranda 🚾 brb
  460. lovetox ok, sorting for incoming order can not work this can be shown with a simple sync example
  461. lovetox 1: write a message
  462. lovetox go offline
  463. lovetox 5: write a message
  464. lovetox now sync 2-4
  465. SaltyBones wait a second, what does incoming order mean? incoming at the server or at the client?
  466. lovetox at the client
  467. SaltyBones but why would you do that?
  468. lovetox because iam the client
  469. SaltyBones yes, but the server already received the message and has an order, right?
  470. lovetox i can not know when something reaches the server
  471. SaltyBones but doesn't the server tell you just this when you query the archive?
  472. lovetox only for the messages that i query
  473. lovetox if i query 2-4
  474. lovetox i get the order for message 2-4
  475. Maranda erm are we talking of re-ordering stuff already in the window? am I getting that right?
  476. lovetox no just db
  477. lovetox so the incoming order in my db is
  478. lovetox 1, 5, 2, 3, 4
  479. lovetox so now i have to sort after timestamps, otherwise i can never recreate the correct order
  480. Maranda You'd need to resort the indexes in the db
  481. lovetox insane Maranda but ok :D
  482. Maranda 1, 5 -> query 2, 4 -> recollimate 1 - 2,3,4 - 5
  483. lovetox how would i know it belongs between 1 and 5
  484. lovetox because of the timestamp
  485. lovetox why should i then reorder indexes
  486. lovetox i could just sort my db with timestamp
  487. Maranda I'd check timestamps if they collide I'd go with the server order for 2,3,4
  488. lovetox so you agree you need timestamps
  489. Maranda thinks he was on that idea from the beginning
  490. Maranda I never said the contrary
  491. Maranda but if there's something colliding I trust the server receive order
  492. Maranda probably needs to make a "learn to communicate" course :D
  493. Maranda probably needs to do a "learn to communicate" course :D
  494. SaltyBones lovetox, but can't you query the server for a UUID that you know?
  495. SaltyBones as in "give me everything after bla and before bla"
  496. Maranda SaltyBones, you can't get an order out of UUIDs though
  497. SaltyBones That might not necessarily fit your desired window but it will allow exact ordering...
  498. SaltyBones Maranda, but if you know the before and after you know where to put the messages...
  499. lovetox but thats not possble with mam
  500. lovetox you can only query timeframes
  501. SaltyBones Although I agree, timestamps or just a counter would be correct in this case.
  502. Maranda I'd just go with the assumption "the server knows better" when there're timestamps collisions
  503. lovetox yes Maranda i do
  504. SaltyBones And then you query a timeframe and you get a bunch of messages with a timestamp that does not correspond to the server side ordering?
  505. Maranda saves you burning neurons for nothing.
  506. SaltyBones that seems unlikely
  507. lovetox but im still wondering if this leaves edge cases
  508. SaltyBones There are timestamp colllisions? wtf...?
  509. lovetox SaltyBones, are you able to write 2 messages in one second?
  510. Maranda lovetox, stop wondering in that case "it's the server fault" that's the ever green that bothers me but let's you sleep lol
  511. lovetox im not saying the server is wrong
  512. lovetox i know that what i get from the server i correctly ordered
  513. lovetox that is the standard ...
  514. lovetox its about what you have inside your DB
  515. lovetox if you get from MAM a message with the same timestamp as you already have in your DB
  516. SaltyBones wait a second
  517. lovetox then you cant say i trust the server, because you didnt get the message in your DB in this mam query
  518. SaltyBones I am always confused about this
  519. Maranda I'd try sorting the indexes in the db, in the manner I told you.
  520. SaltyBones but isn't MAM something that you should basically always query to make sure you don't miss messages?
  521. SaltyBones So you always want "everything since the last message i retrieved"?
  522. SaltyBones Because MAM isn't server side archiving, right?
  523. Maranda so pretend I queried 1,5 if I query 2,4 after I'll always try to stuff that in between 1,5... if it's not possible and the indexes are out of order I'd re-order 'em when I render 'em.
  524. lovetox Maranda, how do you sort them? with what attr
  525. lovetox with the timestamp
  526. SaltyBones lovetox, you could just requery if there is actual ambiguity...
  527. Maranda if timestamps collide I'd then use the order the server sent me, and try to save that in the db (saving microsecond timestamps of receival for example)
  528. SaltyBones although that really should not be necessary...
  529. Daniel Well, it seems that as a discussion-starter.. Lovetox, I will test tomorrow, thanks for the quick fix!
  530. Daniel *was
  531. lovetox reordering messages inside the db is insane
  532. Maranda then don't do it
  533. lovetox this is not viable at all, dbs are way to slow on clients
  534. lovetox nobody does that
  535. lovetox its not necessary
  536. Daniel Other thing: group chat archives are gone for me, or at least the old link isn't working anymore (the one where you can read the room logs in the browser)
  537. lovetox you just sort after timestamp ^^
  538. Maranda leave it out of order, just try to flag messages to avoid collisions and find the right order
  539. lovetox yes Daniel i reported it to Asterix
  540. Maranda I mean if the problem is collisions, just add additional flags client side
  541. lovetox no i think i know my problem
  542. lovetox i imagine cases where you get messages with the same timestamps at different time
  543. lovetox though this will only happen in really rare cases
  544. lovetox mam always gives you all messages with the same timestamp in the same query
  545. lovetox so you know the order of these messages with the same timestamp
  546. lovetox only if for some reason your client crashes or something this would not happen
  547. lovetox probably not worth to think about it
  548. bot Philipp Hörist pushed 1 commit to branch _refs/heads/master_ of _gajim_ < >: *877b754e* < > Simplfy writing to config file - use `with` statement and open() instead of - execute run() on dialog, otherwise sys.exit() is called immediatly
  549. lovetox Wiktor, try tomorrows nightly windows installer
  550. lovetox on your old backup
  551. lovetox i changed some things but im not sure that solves the problem
  552. Wiktor lovetox: sure, I'll try it and report back
  553. lovetox SABer tested on mate
  554. lovetox icon looks good to me in beta-1
  555. lovetox
  556. debacle lovetox, is there a menu on the icon, e.g. to change status?