/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Posting mode: Reply

Check to confirm you're not a robot
Drawing x size canvas

Remember to follow the rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Manage Board | Moderate Thread

Return | Catalog | Bottom

Expand All Images

Version 426 Anonymous Board owner 01/20/2021 (Wed) 23:28:21 Id: f78aa4 [Preview] No. 944
https://youtube.com/watch?v=SpXsBNBtqP0 [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v426/Hydrus.Network.426.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v426/Hydrus.Network.426.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v426/Hydrus.Network.426.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v426/Hydrus.Network.426.-.Linux.-.Executable.tar.gz

I had a great week. I mostly fixed and cleaned things.

8kun board deleted

The 8kun /hydrus/ board is now deleted. I was planning last week to move to Endchan as the new primary, but Codexx of 8chan.moe kindly offered to host me on 8chan.moe /t/, in a new Hydrus Network General. The thread is here https://8chan.moe/t/res/2219.html . The old /hydrus/ board is also migrated to 8chan.moe /hydrus/, if you would like to search it or recover something important to you. All the links across the program and help are updated to reflect the changes. I appreciate the help from Codexx in getting this going. I have never been a comfortable or great BO, so this arrangement is a better fit for my situation for the primary place to talk Anon about Hydrus. Endchan will remain as a bunker, and I will keep making release posts here. Please feel free to use whichever you are most comfortable with.

downloaders from the future

It seems gelbooru changed their markup recently, and our default downloader stopped pulling tags, at least for some. A user helpfully created an update--which is rolled into today's release if you missed it--but unfortunately users who were running older clients ran into an unusual versioning bug. The updated downloader would only work correctly on above v422 or so, leading older clients to get a variety of annoying errors about 'md5', 'hex' or 'additional_info' when either booting or trying to inspect the broken object.

I have fixed the problem here. Anyone who recently imported a 'downloader from the future' should have it fixed today. Furthermore, the various ways you import downloaders now try to check against versions your current client can support, and if they are too advanced, you now get a nice non-spammy popup message and the too-complex objects are discarded from the import. If the advanced object is buried deeper inside the downloader, it may not be able to discard neatly yet, and you will just get a 'soft' popup message about it, but I hope to tighten this up in future.

This version checking is also applied more softly across the whole program. It is less likely that one of your internal objects will be from the 'future', but the client will now make a popup if this is so. Please let me know if you get a whole bunch of these. With luck, this is ultimately a rare problem and I can toughen up this error and actually stop clients from running if their objects ever turn invalid.

I also fixed up error recovery in the downloader system UI and multi-column lists across the program. When some borked object enters them, they now try to show an appropriate 'cannot render' style text, rather than raising an error.

Anonymous Board owner 01/20/2021 (Wed) 23:32:15 Id: f78aa4 [Preview] No.945 del
the rest

I also updated the gelbooru pools downloader. Some users get different pool 'gallery' markup from the site, we are not sure why, but it should be fixed now.

Last week's network object breakup went well overall. There was one unfortunate but harmless error popup that could appear on client shutdown, typically when certain websites had downloads start in the last five minutes of the program, which I have fixed.

Also fixed is the new fast tag search sometimes dropping subtag results. If you noticed that 'samus aran' was not appearing in results, even though 'character:samus aran' existed and could appear in a 'character:anything' query, this is now fixed. Thank you to the users who reported on this and helped me figure it out. A new maintenance routine will run on update to fill in any gaps that may have appeared for you. It shouldn't take more than a minute, even if you sync with the PTR.

I added a new 'profile' mode to the help->debug menu, 'callto profile mode'. This one tracks mostly UI-level jobs that work in the background. If I have been talking to you recently about heavy downloaders or similar making the UI juddery, please give it a go and send me the profile logs as usual. Also, profiles in general should be less spammy with the popups.

The 'bandwidth used this session' section of the main window's status bar is now more accurate. In certain cases, after a delay, it could report bandwidth used in a session in the hours or days before boot, but now it is exclusive to this session. The difference here is usually not noticeable, but if you restarted a client after pausing all network traffic and then left it open for three hours, it could be confusing!

full list

- misc:
- thanks to help from Codexx at 8chan.moe, the old 8kun board is completely migrated and archived at 8chan.moe /hydrus/. going forward I will be maintaining a Hydrus Network General there on /t/ for merged release posts, Q&A, and Bug Reports. the plan is that whenever it fills up, it will be moved to the /hydrus/ archive. the links across the program and help are updated, please let me know if I missed any. Endchan /hydrus/ remains as a bunker
- fixed a bug where subtag entries in the new tag fast search cache were being deleted for all namespaces when a single namespaced version was went to count 0. it meant some autocomplete results were not appearing, often after some sibling changes. a new 'repopulate' job has been added to the database regenerate menu to fix this efficiently if something like it happens again. this routine will be run on update to fix all users, it shouldn't take long (issue #785)
- fixed a bug where the new network objects would throw an error on save when a 'dirty' object was quickly deleted. I think this was typically sessions that only have ephemeral session cookies being created in the final five minutes of the program and then being cleared during program exit

Anonymous Board owner 01/20/2021 (Wed) 23:34:42 Id: f78aa4 [Preview] No.946 del
- when an archive/delete filter finishes, it now fires off all its changes in one go. previously they would go in ~64-file chunks over the next few hundred milliseconds. this will add a small amount of 'refresh lag', delaying page refreshes etc..., on bigger filter jobs for some clients, but it will guarantee that if you hit F5 real quick after finishing filtering on a processing page with non-random sort, you won't see the same files again at the top, only for them to be swiftly archived/deleted as you watch. trash file performance is much better these days, let me know how this goes for you if you do megafilters
- the tag import options whitelist now checks subtags of parsed tags. if you add 'samus aran' to the whitelist, but the site delivers 'character:samus aran', this now passes the whitelist
- thanks to a user's submission, the gelbooru 0.2.5 post parser is updated and should get tags again, for those users who stopped getting them last week--however, I never experienced this myself, so please let me know if you still have trouble. there could be something more complicated going on here
- updated the gelbooru 0.2.x gallery parser to handle an alternate form of gelbooru pools--we did not figure out why different users are being given different markup, it wasn't as simple as being logged in or not, but there is a difference for some. this parser is folded in on update, so the gelb pool downloader should be fixed for users who had trouble with it
- also updated the gelbooru pool gallery url class to infer next page url, as in the alternate form the next page is difficult to parse
- the 'clear all closed pages' command under the 'undo' menu now asks for yes/no confirmation
- added a 'callto' profile mode, which will be very useful in diagnosing GPU lag in future. the 'callto' jobs are little off-main-thread things like image rendering and async panel preparation. should help us figure out where big download pages etc... are eating up CPU
- the different profile modes in the debug menu now all show popup messages, but only when their job exceeds the particular profile's interesting time, usually 3-20ms. this should reduce spam
- the 'this session' bandwidth tracker on the status bar is now a special tracker that only includes data from boot. previously, it was using the 'global' tracker, which after certain time intervals (four minutes, three hours, three days), will compress bandwidth history into larger time windows to save space. if one of these windows covered time before the client started, it could spookily report a little bandwidth used on a client started with network traffic paused
- bandwidth data usage in times shorter than the last ten seconds (which are smoothed out to avoid bumps) now also get the 'don't get bandwidth from the future on motherboards that had a briefly crazy system clock' fix from last week
- fixed some disengaged database tuning that was leading to worse cancel times on certain jobs
- updated a whole bunch of the help so section headings are links with nice #fragment/anchor ids, making it easy to link other users to a particular section. I will continue this work, and future help will follow this new format
- fixed some bad character encodings in the changelog document, siblings help, and tagging schema help. these should now be utf-8 valid

Anonymous Board owner 01/20/2021 (Wed) 23:38:02 Id: f78aa4 [Preview] No.947 del
- object load improvements:
- the client now detects serialisable (saveable) objects that were generated in a future version format your client does not yet support. this mostly affects downloader objects like parsers, where you might import an object a user in a much newer version of the client made. for instance, this week some users imported a fixed gelbooru parser in an older client, which was then saved and double-updated later on, and that caused other problems down the line. downloader imports deal with this situation cleanly, but otherwise it mostly makes a popup notifying you of the problem and asking to contact me. there are about 170 places in the program where objects are deserialised and I am not ready to make this a fullblown error until I know more about people's IRL situations. let's hope this is not widespread. if you run into this, please let me know!
- if you were running an older client and manually imported the updated gelbooru parser that was going around, and then you got errors about 'md5', hex' or 'additional_info' something, it _should_ be automatically fixed on update. you should be able to update from previous to ~422, see it in network->downloader components->manage parsers, and it should just work. many users will have the entry overwritten anyway in the above gelb update I am rolling in. if any of this does still give you trouble, please delete and re-import the affected object(s)
- importing one of these future-versioned serialised objects using the import/export buttons on a multi-column list, either clipboard, json, or png, will cleanly discard future objects with a non-spammy notification
- the Lain drag-and-drop easy downloader import does the same
- the parser 'show what this can parse in nice text' routine now fails gracefully
- multi-column lists now handle a situation where either the display or sort data for a row cannot be generated. a single error popup per list will be generated so as not to spam, bad sorts will be put at the top, and 'unable to render' will occupy all display cells
- .
- network server stuff
- fixed being able to delete an account type in the server admin menu
- the way accounts are checked for permissions serverside now works how the client api does it, unified into a neater system that checks before the job starts
- did some misc server code cleanup, and clientside, prepped for restoring account modification and future improvements

next week

I started on the network updates this week. I will be cleaning more server code and reworking my ancient unit tests, and getting some older admin commands that were 'temporarily' turned off working again. I'll also continue reformatting the help to make sure all the headers have #fragment links, to help external linking to specific sections.

Release Tomorrow! Anonymous Board owner 01/27/2021 (Wed) 09:27:34 Id: 76d418 [Preview] No.952 del
I had an ok week. I mostly fixed and cleaned things. Another instance of 'ghost' pending tags is fixed, with bad counts and tags corrected nice and quickly with a new maintenance routine. Also, the various undocumented shortcuts in the tag autocomplete search box are now exposed and customisable.

The release should be as normal tomorrow.

Top | Return | Catalog | Post a reply