/hydrus/ - Hydrus Network

Hydrus Bunker

Boards | Catalog | Bottom

Check to confirm you're not a robot
Drawing x size canvas

Remember to follow the rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Version 413 Anonymous Board owner 09/23/2020 (Wed) 21:41:54 Id: c8c9d8 [Preview] No. 803 [Reply] [Last 50 Posts]

Message too long. Click here to view full text.

Anonymous Board owner 09/23/2020 (Wed) 21:42:56 Id: c8c9d8 [Preview] No.804 del
- client api:
- wrote a client test for the help menu so I can test some basic functions holistically, hoping to stop some recent typo bugs from happening again
- did a couple of hotfixes for v412 to deal with some client api url pending bugs. the links in the 412 release now point to new fixed builds
- fixed an issue setting additional tags via the client api when the respective service's tag import options are not set to get anything
- fixed a 500 error with /add_tags/add_tags when a tags parameter is an empty list
- fixed the /manage_pages/get_page_info client api help to show the 'page_info' key in the example response

next week

I am now going to write the tag parents database cache. This will basically be the same as I just did for tag siblings, making parents applicable cross-service and completely virtual. I will also improve how the application works, making it pause/resumable in non-interrupting parts in the background, like how repository tag processing works now.

I expect this to take two weeks, possibly three, so please do not expect a release next week. I'll make an update on Tuesday evening as normal.

Anonymous Board owner 09/23/2020 (Wed) 22:15:23 Id: c8c9d8 [Preview] No.805 del
The 'no_proxy' setting may not work quite right, btw. Seems to be some odd IP/domain behaviour. Me and a user are looking at it. If you use a proxy and have any trouble, disable it in settings to go back to old behaviour. If you don't use a proxy in options->connection, nothing has changed, you don't have to do anything.

Parents Cache Update 1 Anonymous Board owner 09/29/2020 (Tue) 22:31:29 Id: 8c4acc [Preview] No.806 del
I had an ok week. I got stuck into the parents cache and related maintenance updates. I have a plan and I am grinding away at it. There is still a lot to do, but current expectation is 414 will be out on October the 14th.

Version 412 Anonymous Board owner 09/16/2020 (Wed) 22:26:27 Id: cc303e [Preview] No. 796 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=3-K94kdbTS8 [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v412/Hydrus.Network.412.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v412/Hydrus.Network.412.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v412/Hydrus.Network.412.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v412/Hydrus.Network.412.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v412.tar.gz

I had a great week catching up on smaller jobs, improving search speeds, and adding a 'lite' 407->408 update mode for HDD users who sync with the PTR. There are also a couple of new applications for the Client API.

Update this week will take a few seconds to a few minutes as new database indices are created.

sibling and search speeds

Message too long. Click here to view full text.

Anonymous Board owner 09/16/2020 (Wed) 22:29:36 Id: cc303e [Preview] No.797 del
new client api applications

A user has been working hard at making a web browser for the client via the Client API, called Hydrus Web. It is now ready at https://github.com/floogulinc/hydrus-web ! If you have a bit of networking experience, please check it out - it allows you to browse your client on your phone!

Also Anime Boxes, a booru-browsing application, is adding Hydrus as an experimental browseable 'server' this week, also through the Client API. Check it out at https://www.animebox.es/ !

I also updated the Client API help to talk more about HTTPS and connections across the internet, here: https://hydrusnetwork.github.io/hydrus/help/client_api.html

full list

- client api:
- added Hydrus Web, https://github.com/floogulinc/hydrus-web, to the Client API page. It allows you to access your client from any web browser
- added Anime Boxes, https://www.animebox.es/, to the Client API page. This booru-browsing application can now browse hydrus!
- the /add_urls/add_url command's 'service_names_to_tags' parameter now correctly acts like 'additional' tags, and is no longer filtered by any tag import options that may apply. that old name still works, but the more specific synonym 'service_names_to_additional_tags' is now supported and recommended (issue #456)
- the /add_urls/add_url command now takes a 'filterable_tags' parameter, which will be merged with any parsed tags and will be filtered in the same per-service way according to the current tag import options.
- the client api help is updated to talk about this, and the client api version is now 14
- updated client api help to talk about http/https

Anonymous Board owner 09/16/2020 (Wed) 22:30:58 Id: cc303e [Preview] No.798 del
- the rest:
- the 407->408 update step now opens a yes/no dialog before it happens to talk about the big amount of CPU and HDD work coming up. it offers the previous 'full' version that takes all the work, and a 'lite' version that applies no siblings and is much cheaper. if you have been waiting on a PTR-syncing HDD client, this should let you update in significantly less time. there is still some copy work in lite mode, but it should not be such a killer
- the 'manage where tag siblings apply' dialog now has big red warning text talking about the current large CPU/HDD involved in very big changes
- a bunch of file-location loading and searching across the program has the opportunity to run very slightly faster, particularly on large systems. update will take a few seconds to make these new indices
- namespace and subtag tag searches and other cross-references now have the opportunity to run faster. update will take another couple of minutes to drop and remake new indices
- gave tag and wildcard search a complete pass, fixing and bettering my recent optimisations, and compressing the core tag search optimisation code to one location. thank you for the feedback everyone, and sorry for the recent trouble as we have migrated to the new sibling and optimisation systems
- gave untagged/has_tags/has_count searches a similar pass, mostly fixing up namespace filtering
- gave the new siblings code a similar pass, ensuring a couple of fetches always run the fast way
- gave url search and fetch code a similar pass, accounting better for domain cross-referencing and file cross-referencing
- fixed a typo bug when approving/denying repository file and mapping petitions
- fixed a bug when right-clicking a selection of multiple tags that shares a single subtag (e.g. 'samus aran' and 'character:samus aran')
- thanks to some nice examples of unusual videos that were reported as 1,000fps, I improved my fallback ffmpeg metadata parsing to deal with weird situations more cleverly. some ~1,000fps files now reparse correctly to sensible values, but some either really produce 1000 updates a second due to malformation or bad creation, or are just handled that way due to a bug in ffmpeg that we will have to wait for a fix for
- the hydrus jpeg mime type is now the correct image/jpeg, not image/jpg, thanks to users for noticing this (issue #646)
- searching for similar files now requires up to 10,000x less sqlite query initiation overhead for large queries. the replacement system has overhead of its own, but it should be faster overall
- improved error handling when a database cannot connect due to file system issues
- the edit subscription(s) panels should be better about disabling the ui while heavy jobs, like large subscription resets, are running
- the edit subscription(s) panels now do not allow an 'apply' if a big job is currently disabling the ui

Message too long. Click here to view full text.

Anonymous Board owner 09/16/2020 (Wed) 22:31:20 Id: cc303e [Preview] No.799 del
- boring details:
- improved a pre-optimisation decision tool for tag search that consults the autocomplete cache for expected end counts in order to make a better decision. it now handles subtag searches and multiple namespace/subtag searches such as for wildcards
- wrote fast tag lookup tools for subtag and multiple namespace/subtag
- fixed some bad simple tag search optimisation code, which was doing things in the wrong order!
- optimised simple tag search optimisations when doing subtag searches
- polished simple tag search code a bit more
- added brief comments to all the new cross joins to reinforce their intention
- greatly simplified the multiple namespace/subtag search used by wildcards
- fixed and extended tag unit tests for blacklist, filterable, additional, service application, overwrite deleted filterable, and overwrite deleted additional
- added a unit test for tag whitelist
- extended the whole 'external tags' pipeline to discriminate between filterable and additional external tags, and cleaned up several parts of the related code
- moved the edit subscription panel asynchronous info fetch code to my new async job object
- cleaned up one last ugly 'fetch query log containers' async call in edit subscriptions panel
- moved the edit subscription(s) panels asynchronous log container code to my new async job object
- misc code cleanup

next week

Message too long. Click here to view full text.

Release Tomorrow! Anonymous Board owner 09/23/2020 (Wed) 01:48:02 Id: f14974 [Preview] No.802 del
I had an ok week. I mostly fixed bugs, aiming for a nice 'clean' release before I go for the parents database cache.

The release should be as normal tomorrow.

Q&A Thread Anonymous Board owner 08/08/2019 (Thu) 00:24:05 Id: 348093 [Preview] No. 10 [Reply] [Last 50 Posts]
Please feel free to ask questions about hydrus here.

As a reminder, the help and getting started guide is here:

101 posts and 6 images omitted.

Anonymous 08/12/2020 (Wed) 11:28:14 Id: 5f6f90 [Preview] No.773 del
Is there any way to select the desired search distance in the duplicate filter for each search? I like having 10 prepared for some cases but it does tend to give me a lot of false positives most of the time.

Anonymous Board owner 08/15/2020 (Sat) 18:03:21 Id: d62469 [Preview] No.774 del
Hmm, I don't think so, I am afraid. Do you mean like when you want to do some dupe filtering, say 'only show me things that are tagged 'blah' and are also exact matches' and such?

At the moment, the actual dupe system does not store search distances that pairs were found at. All that works in the discovery side, which works on a different search system and data structure than the dupe system proper and normal file searches.

If it helps, the likely next step for the dupe system for me is to write a checkbox or something for 'only show pixel-for-pixel-duplicates', as part of a push to early user-customisable auto-resolution rules. Once we can do the easier issue of pixel dupes, I expect the natural next step will be to add search info for very similar dupes (like 'if the files are 99.998% similar, follow these rules').

Anonymous 09/15/2020 (Tue) 16:35:56 Id: 345f8b [Preview] No.794 del
In the PTR, there's a tag that's incorrectly applied to a file. I tried to remove it, but in the "manage tags" window, there's no tag there to be removed. As a test to see if it was a sibling that I just wasn't noticing, I petitioned every tag to be removed (to see the -1 on the tags) and that tag DIDN'T have a -1 next to it. What's going on here?

If this is some sort of moderation thing by the people who run the PTR and the tag is unremovable for some reason, then is there some way that I can locally override that, and have it be removed on my client specifically? I know that the tag is incorrect, so it's a little annoying that I can't fix it.

Anonymous Board owner 09/19/2020 (Sat) 21:44:55 Id: 2dd545 [Preview] No.800 del
Hmm, no, that sounds like a bug. There's no 'pinned' tags system atm.

If you type the tag in the manage tags dialog autocomplete, does it appear to be in any siblings? e.g.:

the ghost tag is "character:samus aran"
you type in "samus"
you see "samus aran (will display as character:samus aran)" in the list somewhere

There appears to be a bug in the new siblings code where sometimes a sibling implication does not remove when the base tag it was implicated by is removed. I am still chasing this down. It seems to occur more often from tag changes by the Client API, but it does not sound like you have been using this to edit tags.

I don't have a good maintenance task to fix a small number of bad files in the case of a miscount, but I will write one when I get to the parents cache in a couple of weeks.

Anonymous 09/21/2020 (Mon) 01:13:52 Id: 345f8b [Preview] No.801 del
The tag in question is "2 females", so it appears in plenty of sibling pairs as the "new" sibling. That's why I clicked "petition all tags" to make sure that it wasn't just that I missed a sibling that I didn't know about being displayed as that tag.

>It seems to occur more often from tag changes by the Client API, but it does not sound like you have been using this to edit tags.

I don't even know what that is, so I don't think I'm doing any of that.

It's good that this is a bug though. I thought it might be a feature. Having tags from the PTR that I can't remove, at least in my own DB, would've been a dealbreaker for using the PTR. I can't have the PTR tags eternally ruining my searches and there being nothing I could do to fix it for myself at least.

Version 411 Anonymous Board owner 09/09/2020 (Wed) 22:15:23 Id: 98d815 [Preview] No. 791 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=NgcYLIvlp_k [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v411/Hydrus.Network.411.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v411/Hydrus.Network.411.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v411/Hydrus.Network.411.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v411/Hydrus.Network.411.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v411.tar.gz

I had a good week. The siblings work is complete.


The last areas that used the old siblings systems are now migrated to the new fast cache. The 'loading tag siblings' step of client boot, which for PTR users could be several seconds of work, is no longer needed!

Message too long. Click here to view full text.

Anonymous Board owner 09/09/2020 (Wed) 22:17:39 Id: 98d815 [Preview] No.792 del
full list

- misc:
- fixed the 'system:(like/dislike) rating = x' search predicate string, which was saying 'unknown' rather than 'like/dislike' in several cases
- fixed a 'current_count' error in the new file search optimisation code for tag searches where the tag did not exist for any files in the domain (i.e. autocomplete count=0). thank you to users for helpful reports here
- fixed the recent file search optimisation code to handle 'system:time imported' when it was mixed with tags or search predicates that would pre-populate the query file pool with file domain cross-referenced files. sorry for the trouble!
- the forced delay overhead for table analysis is reduced from 0.1s to 0.02s. whenever many mostly empty tables need to be analyzed (like on first boot shutdown, when it is usually 100+ tables), it now zips by
- .
- siblings/tag improvements:
- typing a shorthand sibling like 'lotr' into an 'all known tags' 'read' autocomplete - like on a default search page - now reliably discovers and matches text entry to ideal sibling results like 'series:lord of the rings'. this was previously buggy and unreliable--it now allows the match using better db knowledge, even when the merged 'all known tags' services involved disagree on siblings
- when typing tags into a 'searching immediately' page that has media, the autocomplete count results that only refer to that media will now match shorthand sibling inputs to the ideal result. media-populated tag search now takes a little bump of extra CPU to fetch results (they are now passed through the db to get nice siblings info), so it is also cached for the duration of your typing (previously, the counts were re-computed on every new keystroke, so this should be significantly smoother to work with on large pages even if that first keystroke takes a moment to give results)
- when typing into a 'write' autocomplete, like in manage tags, the process that promotes the entry text and known siblings to that entry and a potential ideal sibling result to the top of the list is now more sane. it now also only uses results with nonzero count. we'll see how this last change works out IRL
- when typing into a 'read' or 'write' autocomplete, the pre-search tag insert no longer has sibling insertion/swapping. it was unreliable before, with weird sibling-swapping in the short moment before real results returned. if you have slow results and often quick-add tags into search pages or manage tags, let me know how this works for you
- the 'additional tags' tag input dialog off the tag import options edit panel now shows the 'will display as' label
- the 'favourite', 'file lookup', and 'recent' tag suggestion panels now show the 'will display as' label
- the 'related' suggestion panel, which works on a slightly different system, now shows the 'will display as' label
- the 'tag suggestions' options panel's 'favourite tags' edit lookup and list now displays 'will display as' labels and correctly finds service-specific siblings in its results (e.g. you type 'lotr', it also finds 'series:lord of the rings')

Message too long. Click here to view full text.

Anonymous Board owner 09/09/2020 (Wed) 22:18:18 Id: 98d815 [Preview] No.793 del
- the last UI-side siblings work is cleared:
- the UI-side tag siblings cache is no longer used. the sometimes multi-second 'loading tag siblings' step of boot no longer happens!
- media autocomplete fetches are now asynchronously populated with siblings data via the db
- the exact-match and sibling 'insert' predicates at the top of pre-load and post-load read and write autocompletes now rely exclusively on db data for sibling matching
- taglists now present 'will display as' labels asynchronously and are better about updating those labels when the list's underlying tag service changes
- the taglist right-click menu that shows siblings to copy now fetches that submenu's contents asynchronously from the database
- the test panel on a blacklisting tag filter now asynchronously fetches tag siblings to test against from the database
- the actual blacklist tag filter test now fetches tag siblings to test against from the database
- reworked my custom tag listbox to handle asynchronous text decoration, and unified sibling decoration for media taglists and string taglists
- updated my old async updater class to be more flexible for different job types, and cleaned the code that already used it
- wrote a simple class for one-shot async jobs
- wrote a simple db lookup for UI-side tag sibling chain members
- wrote a simple db lookup for UI-side tag ideal siblings
- bunch of misc sibling, db, and ui work and cleanup to make all this work

next week

Message too long. Click here to view full text.

Release Tomorrow! Anonymous Board owner 09/15/2020 (Tue) 23:38:26 Id: 8d4d4c [Preview] No.795 del
I had a great week. I was able to catch up on smaller work, optimised a variety of db functions, including the new sometimes-slow tag and wildcard queries, and added a 'lite' mode for the big 407->408 update so PTR-syncing users on HDDs can update in less time.

The release should be as normal tomorrow.

Version 410 Anonymous Board owner 09/02/2020 (Wed) 22:41:28 Id: a6ea8b [Preview] No. 787 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=N7GaWASGcgs [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v410/Hydrus.Network.410.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v410/Hydrus.Network.410.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v410/Hydrus.Network.410.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v410/Hydrus.Network.410.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v410.tar.gz

I had an ok week. I wasn't as productive as I hoped, but I am happy with the mostly optimisation work.


After some more profiling in IRL situations, and with more helpful info from users, I have done another round of profiling for the new sibling cache, and more besides. A database technique I use for many purposes is now more reliable (fewer lag spikes), and has less CPU overhead. If you found some systems (like the 'related tags' suggestions in manage tags dialog) sometimes took a few seconds to work in the past couple of weeks, they should now be fast again. And you should find many types of file search, particularly those with multiple search predicates, and general tag processing, should be faster than before.

Message too long. Click here to view full text.

Anonymous Board owner 09/02/2020 (Wed) 22:41:53 Id: a6ea8b [Preview] No.788 del
full list

- general work:
- fixed a bug in the new file service filtering code that was stopping file upload commands to file repositories or ipfs services from sticking
- fixed an issue with the export files dialog auto-close-when-done function
- I think I fixed a possible bug in the boot file location repair/recovery dialog sometimes not saving corrected paths on unusual file systems
- file migration cancel button and shut off timer should work a bit more reliably, more to come here
- copying subscription quality csv info to clipboard no longer does nice human numbers (you now get 1234, not csv-breaking 1,234)!
- may have fixed a very rare 'or predicate' error when opening a dialog with a 'read' autocomplete input, like export folder or file maintenance jobs dialogs
- all pages are better about dealing with missing (i.e. recently deleted) services on load, and autocompletes also
- error handling from servers with strange character encodings should be better about dealing with null characters
- cleaned up the combined display regen chain code
- deleted some obselete db code
- .
- optimisation review:
- after more profiling, and thanks to additional input from users, I have done another round of optimisation for the new caches. using a new technique, more than just mappings are sped up this week - a number of queries that were prone to lag spikes should now have much more reliable speed and also be faster when hammered often
- .

Message too long. Click here to view full text.

Anonymous Board owner 09/02/2020 (Wed) 22:42:17 Id: a6ea8b [Preview] No.789 del
- other optimisations:
- mappings processing
- sibling processing
- wildcard tag searches, with and without namespaces, particularly when mixed with other search terms
- 'tag as number' searches, with and without namespaces, particularly when mixed with other search terms
- searching for tags when mixed with other search terms
- has notes/no notes
- searching files on 'all known files' with general file metadata system predicates (like size, filetype)
- url class, url domain, and url regex file searches, particularly when mixed with other search terms
- num tag file searches when mixed with other search terms
- has/not has tags file searches when mixed with other search terms
- sped up specific display chain regen significantly, with similar separate current/pending optimisations as last week's for combined
- converted specific display cache overall regen to use a copy followed by the new chain regen rather than additive file import
- sped up combined display chain regen a little bit
- the splash window now updates itself with less UI overhead, so spammy updates (like the new tag regen code) use a little less CPU and fewer UI context switches

next week

Message too long. Click here to view full text.

Release Tomorrow! Anonymous Board owner 09/08/2020 (Tue) 22:49:25 Id: b43780 [Preview] No.790 del
I had a good week. I fixed some recent search bugs and I capped off the new siblings work. Everything siblings is now running off the new cache, so the slow 'loading tag siblings' step of boot no longer occurs!

The release should be as normal tomorrow.

Version 409 Anonymous Board owner 08/27/2020 (Thu) 01:30:28 Id: deeceb [Preview] No. 783 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=puU4EvBTPxA [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v409/Hydrus.Network.409.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v409/Hydrus.Network.409.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v409/Hydrus.Network.409.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v409/Hydrus.Network.409.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v409.tar.gz

I had a great week fixing some bugs and optimising the new tag siblings cache. The new code works much faster now.


I am very happy that there do not seem to have been any obvious errors with the new sibling database cache. Unfortunately, a couple of areas were working inefficiently, which IRL testing helped to diagnose. I put a lot of time into this this week and was very successful - some sections take 10% less time, some 90%, and one critical query now takes 99% less time. It depends on many factors, but many things are faster overall. In particular, tag processing speed, which took a real hit, is back up to good speed, and setting new tag sibling application rules now only needs to regenerate for changed siblings, so if you add (or remove) your own five 'my tags' siblings onto the PTR, the client now only has to do two seconds of work, not ten minutes.

Message too long. Click here to view full text.

Anonymous Board owner 08/27/2020 (Thu) 01:32:06 Id: deeceb [Preview] No.784 del
full list

- siblings:
- the slowest of the new sibling regen & update code has received a full optimisation pass. some sections take 10% less time, some 90%, and one critical query takes 99% less time. overall, several big jobs work much faster, and ptr processing, which slowed significantly for many users, should be back up to a good speed. uploading pending tags (which tend to be for local files) should be much faster in particular. let's do another round of IRL observation and profiling this week, and I'll keep at it
- the various 'display' regeneration routines now provide more progress status text, drilling down to the x/y siblings being collapse-counted, or number of files added to a cache, and generally all tag sibling regen got a status update polish pass
- optimised the way tag sibling application is set--now, only the tag siblings that are changed need to have their counts regenerated. hence, if you just apply (or remove) your own five 'my tags' siblings onto the PTR, the client now only has to do two seconds of work, not ten minutes
- .
- the rest:
- fixed the annoying issue with media viewer mouseovers stealing focus/activation from the manage tags dialog. this can now only happen if current focus is on a hover window. sorry for the delay!
- updated manage tag parents dialog to state the pairs being petitioned on the 'petition reason entry' dialog
- updated manage tag parents and siblings dialogs to have appropriate 'reason' suggestions for petitions (previously, they were inheriting the same suggestions as for add)
- ipfs network jobs now have a minimum 'reply' connection timeout of two hours (so giganto directory pushes won't throw an error). connection timeout remains the same, so if the server is hanging on that, it'll still notice
- fixed the 'test address' button on the IPFS manage services panel
- petitioning an IPFS file when there is no IPFS multihash entry in the db no longer causes an error. now, in this case, the file entry is removed with no change made.
- when pending to or petitioning from a file service, a quick filter is now applied to discard invalid files (i.e. (not) already in the service). any weird logical holes where this might occur should now be fixed
- export folders now catch and report missing file errors more nicely
- export folders now remember the last error they encountered and report that in the edit export folders dialog

Anonymous Board owner 08/27/2020 (Thu) 01:32:35 Id: deeceb [Preview] No.785 del
- boring tag siblings optimisations:
- optimised the tag manager generation routine to use any common file domains for fast cache lookup for any subset of the files available, rather than falling back to 'all known files' domain when there is no single common file domain
- optimised the new 'all known files' display autocomplete cache to use similar faster specific files cache lookups when available
- optimised how the 'all known files' display cache regenerates tag sibling chains. it now takes a shortcut when given non-sibling tags and tags where all but one sibling member have zero count, and it can count current and pending counts separately according to the most efficient counting method (e.g. most pre-display pending counts are 0 across the board, so even if current count is a million, the pending count can often be assumed without lookup overhead). furthermore, the 'clever' count has better query planning and less non-sqlite data overhead, and with experimental data is now chosen more carefully. what was previously a 22s job on a test database now takes 5s
- deduplicated how new mappings are filtered to all the specific cache domains, significantly reducing overhead
- massively optimised a critical - and the slowest - part of the new 'combined' cache that handles add/pend mappings pre-insert presence testing, speeding up the core query about 100x!
- reduced some overhead when doing file service_id normalisation in repository processing
- split up specific chain regen into groups to reduce memory usage
- optimised specific display tag cache 'add file' updates, and thereby basic cache regeneration, to be just a little faster for files that have multiple sibling tags
- all predicates made in the database are now populated with ideal and chain sibling information, and this is used for '(will display as xxx)' labels and autocomplete tag search filtering (e.g. you type in 'lotr', it matches an autocomplete result of 'lord of the rings'). there are still some ui-made predicates to figure out, so the old system remains as a fallback
- related tags lookup is a tiny bit faster and now populates its predicates with ideal and chain sibling info at the db level
- cleaned up some 'fetch related tags' code, might make it a bit faster for large tag counts
- cleaned up the way some mapping tables are fetched
- unified table/table_name nomenclature in the db code
- updated an old data->ui status presentation method (it typically does stuff like "regenning some stuff: 500/10,000"), to not hog so much UI time and not yield worker threads so often when new statuses are coming in real fast
- several late optimisations based on IRL testing

Message too long. Click here to view full text.

Release Tomorrow! Anonymous Board owner 09/02/2020 (Wed) 05:40:55 Id: deeceb [Preview] No.786 del
I had an ok week. I wasn't as productive as I hoped, but I fixed some bugs and made great progress in optimising the new sibling code, and older code too. The various lag spikes in systems like 'related tags' should be smoothed out, many file searches are snappier, and tag processing should be faster than it was in v407.

The release should be as normal tomorrow.

Version 408 Anonymous Board owner 08/19/2020 (Wed) 23:03:41 Id: 39340f [Preview] No. 776 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=X8zLP-TBLcA [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v408/Hydrus.Network.408.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v408/Hydrus.Network.408.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v408/Hydrus.Network.408.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v408/Hydrus.Network.408.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v408.tar.gz

I had a great couple of weeks moving tag siblings forward. The update this week will take some time as a new cache is generated. On an SSD with the PTR synced, about 10 to 20 minutes, depending on how many files you have. Some of the new tag sibling application rules may have 5-10 minute delays on edit as well. If you have a large/PTR client and are on an HDD, or you would rather just wait for the kinks to be ironed out, you might like to wait a week or two for me to further optimise this new code.


tl;dr: Siblings are faster and better now, you don't have to do anything. Some parents will not appear with new downloads - don't worry about it, they will all fill back in nicely soon.

Message too long. Click here to view full text.

1 post omitted.

Anonymous Board owner 08/19/2020 (Wed) 23:07:24 Id: 39340f [Preview] No.778 del
full list

- tag siblings cache:
- tl;dr: siblings are faster and better now, you don't have to do anything. some parents will not appear with new downloads - don't worry about it, they will all fill back in nicely soon
- wrote the first version of a 'tag display' cache, which stores not your tags as they are, but how they appear after display rules such as siblings, parents, and filtering are applied, meaning this data need not be calculated every time on thumbnail load. this week marks the first concrete step forward in an improvement of siblings and parents storage, and begins with just siblings. all siblings and front-end tags work should be generally faster and more accurate
- part one is for tag domains cross-referenced with file domains. it maintains virtual sibling-collapsed mappings and autocomplete counts through mappings added, deleted, pended or pend-rescinded, files added/deleted, and siblings added/removed
- part two is for tag domains on the 'all known files' domain (i.e. no file domain). it maintains virtual sibling-collapsed autocomplete counts through mappings added, deleted, pended or pend-rescinded, and siblings added/removed
- both parts also support full table drop/regen (under the new database->regenerate->tag display mappings cache) for when my logic inevitably miscounts something. the existing regen 'tag mappings cache'/'tag siblings lookup' commands also regen the display mappings cache, since it relies on them
- when tag siblings on a repository are petitioned to be deleted, they are now instantly discounted from tag sibling application (previously, they had to be uploaded and committed to count, now both pending and petitioned offer a quick preview of outcome)
- the display cache supports the tag sibling service application rules under the 'services menu', and regen when that changes, so you can now turn siblings on and off, and apply them across services. as a result, the old 'apply all siblings to all services' option is now gone! as parents will undergo a similar change soon, and the siblings changes this week may lead to some undesired parents in the interim, the 'apply all parents to all services' option is also gone
- tag autocomplete counts in the form (x-y) due to siblings are eliminated. it will still do it when combining the same merged tag across different services, or when an unnamespaced tag includes how many potential namespaced will also be found
- the following search types now obey tag sibling application rules accurately: number of tags search, namespace:anything search, wildcard search, tag search (on a per-tag-domain basis, previously it was globally hacked to all siblings), tag-as-number search. for instance, if you search series:anything, a file that has 'metroid' tag-siblinged to 'series:metroid' will now correctly appear.
- the above search types are now exact to how the tag displays. if you have for files that are tagged 'samus' on either tag service A or B, and service B has a sibling for that to 'character:samus aran', searching for 'samus' gets the results in A, 'character:samus aran' gets the results in B. previously it was an expensive logical mish-mash of 'sure, try and get everything behind the scenes'. now it searches what you see
- searching for files in the advanced 'all known files' domain currently has no sibling support for the above search types. autocomplete counts should be good, and the results that come up should have the correct tag display, but the actual results are calculated based on storage tags. getting this to work without doubling the size of the db is tricky, so it will have to be ongoing work

Anonymous Board owner 08/19/2020 (Wed) 23:07:56 Id: 39340f [Preview] No.779 del
- all tag siblings are now completely virtual. this means that when a tag comes in via a downloader or other means, it will not be automatically coerced to its ideal sibling in the actual db storage tables (the true tags you see in manage tags dialog), but remain as it is. there is no change in sibling appearance in normal operation--it still _displays_ and searches as its ideal sibling. the same will happen to parents in the coming months, and in the interim period, parents no longer apply across siblings. as siblings can come and go from anywhere, they are now divorced from actual stored tag mappings. in a similar way, the manage tags dialog no longer supports the 'hard-replace siblings and parents' command, nor the 'auto-replace with sibling' command. this may be jarring to workflows and preferences, so please bear with me and let me know what feels particularly bad. and please don't worry too much about parents not always being added in the meantime--I hope to do the same transition for them in four of five weeks, and all gaps will be filled in. also, in the coming weeks, I expect to improve manual tagging workflow by indent-grouping edit-view siblings together (ditching the old 'will display as' text) for easier review and selection, a bit like parents. actual 'hard' siblings and parents that do always get irreversibly renamed/added in storage will come in the future as a separate system
- the 'add_siblings_and_parents' client api parameter no longer adds siblings, and soon will be retired completely
- I had wanted to completely eliminate the old UI level siblings manager this week, but there are still some systems, mostly tag autocomplete work, that need it and are tricky to swap. I stripped it down, at least, and reduced its update delay to 2 seconds. therefore, the 'loading tag siblings' step of boot still occurs, albiet a _little_ faster. I hope to have it gone soon
- this is some complicated code affecting core systems. almost everything 'siblings' is now different. there are likely to be laggy parts, awkward new workflow, and possibly some update or miscount bugs as I iron it out. the good news, now they are all virtual, is that problems are undoable. please report any issues, and I will work on it as I keep pushing on this and on parents
- please expect your client.caches.db file to expand in size about 10-30% or so this week. the update itself will take a few minutes as the improved tag lookups and new caches are regenerated from empty
- .
- boring mostly db optimisation list:
- after some thought, moved those new options for tag sibling application down to the db. previously, they were stored in an UI object for convenience, but since everything is going down to the db, it is worth doing it properly down there. thus they reset this week to the default
- I also removed that complicated 'all known tags' page in the tag sibling application options--it wasn't doing enough to justify itself

Anonymous Board owner 08/19/2020 (Wed) 23:09:05 Id: 39340f [Preview] No.780 del
- tag siblings lookup cache now obeys the tag sibling application rules and regenerates the appropriate cache when the options change
- tightened up the db tag siblings lookup cache and wrote more tools for it. it had a couple of blind spots for getting all siblings in a chain. also optimised the lookup for en-masse tag operations
- tightened up my tag sibling structure builder object, which was not eliminating loops but collapsing them to (generally harmless, but not desireable) (A,A) pairs
- extended mappings and siblings lookup caches to perform various sorts of tag sibling collapse filtering, to determine files that do or do not have another tag mapping on a tag sibling chain
- optimised the existing mappings cache in several ways
- optimised cross-domain file cache mappings filtering, and cleaned the code
- optimised autocomplete count fetching from the mappings cache, particularly for large result sets
- optimised how the combined autocomplete count generates from nothing
- optimised how tags are loaded for search results (thumbnails)
- optimesed basic tag search
- greatly optimised how the mappings caches request cross-domain file cache filtering
- broke up the rescind_pending/add mappings job into simpler separate parts, which was needed for accurate display cache counting. this may end up fixing the other weird pending miscount bug we had
- the cached 'display' tags are now loaded with regular media results, not generated on the fly on first request (unless in the advanced 'all known files' domain, where it is done quickly on first load at the db level)
- converted the db over to using its local sibling lookup cache for all sibling jobs
- all data-level content updates to media result objects now occur in the database loop, reducing lag and eliminating a single UI event loop gap when the objects the UI relies on were desynchronised
- optimised how the tag and hash id-to-definition cache maintains itself
- cleaned up cache code generally

Message too long. Click here to view full text.

Anonymous Board owner 08/19/2020 (Wed) 23:09:50 Id: 39340f [Preview] No.781 del
- the rest:
- fixed the stupid manage tag siblings dialog input/ok bug I introduced last week
- fixed the pair preview label in manage tag siblings dialog when it asks to enter a reason for a remove petition
- I believe I fixed the annoying recent bug where the top-right hover window would sometimes not position itself correctly on a window size/move until the top hover was shown once
- fixed a bug where the 'do you want to do shutdown work?' dialog was not abandoning shutdown if cancelled (rather than yes/no)
- updated the 'has free space to do db transaction?' checker, which needs to test device partitions, to do two sweeps--first only fast local devices, then potentially mega laggy network discovery if the mount point is not found (hydev was wondering why it was suddenly taking nine seconds to close his test client!)
- fixed another issue with double-clicking some addremove/queue listboxes when no edit button is set--now in this case they all delete on a double-click
- fixed a little bad error handling on pending content upload. an error with petitioning certain IPFS uploads is not yet fixed

next week

I was unable to completely replace the UI-side siblings manager. It is a bit smaller and faster, but you will still get the slow 'loading tag siblings' on boot. There are about four or five things it still does that I still need to move down to the new cache, so I will keep hammering away. I'll also improve the speed of these new caches' regen. Other than that, I have two weeks of other jobs to catch up on, so just some misc work.

I just realised I forgot to fix the annoying manage tags/media viewer focus stealing! I'll fix it for 409!

Release Tomorrow! Anonymous Board owner 08/26/2020 (Wed) 03:10:42 Id: 9a3e25 [Preview] No.782 del
I had a great week fixing some bugs and making the slowest parts of the new sibling cache code work much faster and more efficiently. The lag that was introduced last week should be reduced, I hope, by about 4-10 times, making it much less noticeable. The UI feedback on longer jobs is also improved.

The release should be as normal tomorrow.

Version 407 Anonymous Board owner 08/05/2020 (Wed) 21:37:04 Id: 99495e [Preview] No. 765 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=3Rup3EdA0kw [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v407/Hydrus.Network.407.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v407/Hydrus.Network.407.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v407/Hydrus.Network.407.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v407/Hydrus.Network.407.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v407.tar.gz

I had an ok week. I did some prep work for siblings, and fixed some bugs.


The right-click menu off a normal taglist now provides a whole bunch of different copy options, depending on whether you want all or selected, tags or subtags, and with or without counts.

Message too long. Click here to view full text.

1 post omitted.

Anonymous Board owner 08/05/2020 (Wed) 21:41:03 Id: 99495e [Preview] No.767 del
full list

- sibling prep:
- I am preparing for a new siblings database cache for v408. this will ultimately make siblings (and parents) faster, more accurate, more powerful, and simple to undo. I have decided, as part of it, to make siblings and parents completely virtual (i.e. the tags won't exist for real, they'll be implied). better tools to manage hard-replace siblings and parents will come later, as trying to support both situations at once has not been excellent
- .
- created options to hold per-service sibling and parent preferences, so you'll be able to set up '"my tags" siblings and then "ptr" siblings apply to "my tags"' or 'no parents apply to this service'
- wrote UI for the sibling options under 'services->manage where tag siblings apply'. you can play with it if you like, and it saves values, but it is not plugged in yet and makes no changes
- siblings logic is a little tighter. the db and gui side of siblings structure calculation is more unified, petitioned siblings are discounted properly on all generation, and the db side now resolves conflict decisions the same on every regen. the gui-side still runs on an older structure, but will be updated to exactly mirror the db
- updated and unified how large numbers of raw tag siblings are fetched in the database. it also supports fast tag slicing, speeding up sibling cache maintenance. the siblings lookup cache now uses this method for regeneration and update calls
- .
- the rest:
- tag right-click menu copying now supports all combinations of selected/all, tags/subtags, and no_count/with_counts where appropriate (issue #325)
- if the media viewer is too thin for the top hover window to fit into its space, the top-right hover now drops down below it. I don't really like how this looks, and will probably instead figure out a flow layout so the toolbar buttons always fit, but at least they are now accessible (issue #388)
- altered the above fix--if the top-right hover window can be shrunk to fit in the available space, it will now squeeze in, only bumping down if it can't
- moving the mouse off an activated (e.g. clicked) hover window now instantly activates the main canvas. this should fix up some fast swallowed clicks and annoying click-to-activate issues with the center-right duplicates hover window, which does not hide (issue #384)
- the duplicates hover window now positions correctly if its min size is too wide to fit in a thin media window
- if you make changes to a parser or content parser, there is now a yes/no confirmation when trying to cancel the dialog

Message too long. Click here to view full text.

Anonymous Board owner 08/05/2020 (Wed) 21:41:34 Id: 99495e [Preview] No.768 del
next week

So, I will try to get this cache working. I need to essentially mirror the existing tag and autocomplete caches, but after siblings collapse, ensure that is maintained through tag or sibling changes, and then plug it in to everything. There is a bunch of ancient code to update around here, and other auxiliary system to improve, so this may end up being a two week job. I'll update on Tuesday night as normal.

Anonymous Board owner 08/08/2020 (Sat) 17:57:13 Id: 99495e [Preview] No.770 del
Hey, I messed up the siblings dialog in 407, it won't ok correctly. I'll fix it all up in the bigger siblings work in 408.

No Release Tomorrow! Anonymous Board owner 08/11/2020 (Tue) 23:01:10 Id: 99495e [Preview] No.772 del
I had a good week getting into the meat of the tag siblings database work. There is a lot to do, but I have a plan I am happy with, it is going well, and I am confident I will have a good first version ready for next week, Wednesday the 19th.

So, no release tomorrow!

Release Tomorrow! Anonymous Board owner 08/19/2020 (Wed) 07:08:20 Id: 0798bc [Preview] No.775 del
I had a great couple of weeks moving tag siblings forward. The client now handles siblings more quickly and accurately using a new database cache. They are easier to undo going forward, and you can also choose where siblings apply on a per-service basis (e.g. turning off the PTR's default siblings and using your own instead).

The release should be as normal tomorrow.

Version 406 Anonymous Board owner 07/29/2020 (Wed) 21:24:33 Id: a9d519 [Preview] No. 760 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=UZr6b4mBoxI [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v406.tar.gz

I had a good week. It is now easy to deduplicate queries in subscriptions.


Managing multiple subscriptions can get complicated. Figuring out which queries are in which sub, whether and where you have duplicates, is tedious. A recent bug with query pasting also could have introduced some duplicates within the same sub. This week should make it simple to manage.

Message too long. Click here to view full text.

Anonymous Board owner 07/29/2020 (Wed) 21:24:57 Id: a9d519 [Preview] No.761 del
full list

- subscription management:
- the manage subscriptions dialog now has a 'deduplicate' button. it is enabled whenever your subs of a particular downloader contain duplicate queries. it launches a semi-bananas but thorough 4-step process to ask if you want to do upper/lower-case deduplication, then which downloader, then which queries, then which master sub(s) to retain the queries
- subscription dedupe within the same sub keeps the query with the most files
- the manage subscriptions dialog also now has a 'lowercase' button that coerces all queries of the selected subs to lowercase
- when pasting a list of queries into a subscription, the 'already in sub' test is now caseless. pasting "Samus_Aran" into a sub already with "samus_aran" will not add anything
- .
- misc:
- url classes now have a checkbox to keep fragment data (e.g. "#kwGFb3xhA3k8B") during URL normalisation. this data is not sent to the server and is not useful for almost all sites, but for sites like Mega, it contains useful clientside javascript navigation or access info if you open the URL in your browser
- fixed video resolution parsing for some unusual SAR files. this stretches a video slightly, usually when the vid was created or converted with older analog tech (e.g. NTSC)
- fixed rating system predicate label for 'rated/not rated'
- the issue where miscounts in pending upload data would persist, sometimes leading to an annoying 'pending (13)' style menu that would not clear without debug action, is now fixed in a cheap way. on any upload action, this cached count is reset. a fix for the actual unusual miscount situation will have to come later
- the different in-memory manager objects now save changes at different time intervals--lightweight things like favourite searches still save not long after any change, but column widths, network sessions, and bandwidth use now save only every ten minutes
- I _may_ have fixed an issue with favourite tags not sticking correctly or resetting when added en masse via the tag right-click menu
- I believe I fixed a rare but permanent ui hang on highlighting a gallery or watcher when that same downloader was spamming through a largely 'already in db/previously deleted' list
- copying tags 'with counts' now works correctly for simple tag views (previously, it only worked for 'predicate' views)

Message too long. Click here to view full text.

Simple Release Tomorrow! Anonymous Board owner 08/05/2020 (Wed) 00:54:39 Id: f168ce [Preview] No.763 del
I had an ok week. Some of the work I did (siblings improvements) is not ready to be turned on yet, so tomorrow's release mostly just has bug fixes, including several related to the media viewer's hover windows positioning and behaviour.

The release should be as normal tomorrow.

Version 405 Anonymous Board owner 07/22/2020 (Wed) 22:04:50 Id: b79965 [Preview] No. 756 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=6e5yyIcq40o [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v405.tar.gz

I had a good week. 'system:number of tags' now supports namespace filtering.

number of tags

The 'system:number of tags' predicate now lets you attach a namespace, so you can search for 'files with two character tags' or 'files with more than one creator', or any other combination you can think of. Also, all 'number of tags' queries have been optimised, and are now typically much faster, particularly when mixed with other tags. They are also quickly cancellable, so if you do enter a very slow one, it should respond fairly quickly if you hit the 'stop search' button that appears after a delay.

Message too long. Click here to view full text.

Anonymous Board owner 07/22/2020 (Wed) 22:06:44 Id: b79965 [Preview] No.757 del
ptr parents

Please note that the public tag repository is turning off tag parent submissions for regular users for a bit. If you haven't seen it already, the 'ptr' tab in manage tag parents will disappear in a few days. The task of cleaning up old and ongoing mistakes is proving too awkward with the current tools, so it is halted until I have a 'virtual parents' rework done, which will allow for more cleanly undoable parents (and hence less contagious bad ones). If you have seen 'shadow the hedgehog' or other bizarre tags appearing in odd places, this is what we hope to fix.

I have scheduled my next 'medium-size' job week, 408, to be for a 'presentation' tag cache, which will allow fast and accurate searching and loading of the tags you actually see on the front end. It will start with siblings, extend to tag filtering/censorship, and then parents will be made virtual and moved to it as well. This was a priority at the end of last year, before Qt and 2020's fun appeared, so I would now like to focus on it again.

full list

- tag search:
- system:number of tags now supports namespaces, for example 'find files with two character tags'! (issue #280)
- it also supports wildcard namespaces, as now do regular namespace search predicates. both run faster. "crea*:anything" is now possible
- system:number of tags has been optimised, and in many cases is now ten to a hundred times faster
- system:number of tags still does not support siblings, something I hope to start correcting as of v408
- both tag existence (numtags =0 or greater than 0) and tag count database routines now respond quickly to 'cancel search' commands, so if you do run a slow query (a bare 'has creator tag' search on 'all known files' on the PTR, for instance), you can now back out quickly after the 'stop' button appears
- note that 'system:number of character tags greater than 0' and '= 0' are equivalent to +/-character:anything, which will be swapped in if you enter these. also, +/-unnamespaced:anything can now appear
- the program is a bit better about determining =0 and greater than 0 and less than 1 being 'none' and 'any but none', when it needs to determine optimisations and special labels
- unfortunately, I am taking away the default value for system:num tags in the options page (edit: I am killing the whole panel now). this old ugly mess of stacked predicate edit panels works on ancient, difficult to update code, so I will retire it and replace it with a unified system that is easy to use, supports in-search system predicate editing, and keeps up with changes automatically

Message too long. Click here to view full text.

Anonymous Board owner 07/22/2020 (Wed) 22:08:24 Id: b79965 [Preview] No.758 del
- database repair:
- the database menu has a new entry, 'repopulate truncated mappings tables', under the newly renamed 'check and repair' submenu, which will try its best to 'fix' a client.mappings.db file that has been truncated due to hard drive fault by repopulating from the local-file-only tag cache. do not run this unless you know you need to
- the 'help my db is broke.txt' document has a full update pass. the language is clearer, common issues and questions are better addressed, two new recovery routines are added, a section on the stages after boot recovery (like the new repopulate job above) is added, and I added my stock 'now become a backup patrician' nag at the end
- the debug routine to clear cached service info numbers is now moved to the 'regenerate' database menu. this thing fixes hanging incorrect 'pending' counts until I can fix it properly
- .
- the rest:
- fixed an issue where when you pasted queries into a subscription, those that were already in the sub (and got the dialog saying so), were being added anyway! I believe this bug came in the last few weeks, after the data storage rewrite. please check your pasted-into subs for dupes
- fixed tab double middle-click behaviour (so you can spam page close), which I thought I had fixed last week but actually messed up completely right at the end (issue #314)
- cleaned up some more of the page tab event code--it was a mess all around. should all be on Qt now, no wx hacks
- network jobs will no longer wait for and consume bandwidth start tokens while all network traffic is paused. all bandwidth competition now halts. (previously, they would continue to consume tokens according to current rules and then all rush to start as soon as traffic was resumed)
- fixed some client booru/client api requests to correctly 404 on missing file results, rather than 500
- cleaned up some file sort code and fixed the sort string conversion, which was rendering the opposite sort direction (asc/desc) in summary labels (e.g. on manage favourite searches)
- cleaned up some ui layout stretching code, including some borked tag import options expand sizing
- improved some button and padding layout definitions, and improved, slightly, the way the top-right media viewer hover window lays itself out and changes its size on media change
- improved some review services layout. should be fewer weird heights and widths in unusual situations, and the new multi-column list fits better
- the manage subs dialog now saves its changes to db more cleanly and atomically
- updated the default derpibooru parser to pull species tags. ten points if you can guess what that is most of the time

Message too long. Click here to view full text.

Release Tomorrow! Anonymous Board owner 07/28/2020 (Tue) 23:46:50 Id: 0286c0 [Preview] No.759 del
I had a good week. Along with some simpler code cleanup, I fixed some bugs, added some quality of life, and made it easy to detect and remove duplicate subscription queries, even across different subscriptions.

The release should be as normal tomorrow.