/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Boards | Catalog | Bottom

Check to confirm you're not a robot
Drawing x size canvas

Remember to follow the rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Version 405 Anonymous Board owner 07/22/2020 (Wed) 22:04:50 Id: b79965 [Preview] No. 756 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=6e5yyIcq40o [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v405.tar.gz

I had a good week. 'system:number of tags' now supports namespace filtering.

number of tags

The 'system:number of tags' predicate now lets you attach a namespace, so you can search for 'files with two character tags' or 'files with more than one creator', or any other combination you can think of. Also, all 'number of tags' queries have been optimised, and are now typically much faster, particularly when mixed with other tags. They are also quickly cancellable, so if you do enter a very slow one, it should respond fairly quickly if you hit the 'stop search' button that appears after a delay.

Message too long. Click here to view full text.

Anonymous Board owner 07/22/2020 (Wed) 22:06:44 Id: b79965 [Preview] No.757 del
ptr parents

Please note that the public tag repository is turning off tag parent submissions for regular users for a bit. If you haven't seen it already, the 'ptr' tab in manage tag parents will disappear in a few days. The task of cleaning up old and ongoing mistakes is proving too awkward with the current tools, so it is halted until I have a 'virtual parents' rework done, which will allow for more cleanly undoable parents (and hence less contagious bad ones). If you have seen 'shadow the hedgehog' or other bizarre tags appearing in odd places, this is what we hope to fix.

I have scheduled my next 'medium-size' job week, 408, to be for a 'presentation' tag cache, which will allow fast and accurate searching and loading of the tags you actually see on the front end. It will start with siblings, extend to tag filtering/censorship, and then parents will be made virtual and moved to it as well. This was a priority at the end of last year, before Qt and 2020's fun appeared, so I would now like to focus on it again.

full list

- tag search:
- system:number of tags now supports namespaces, for example 'find files with two character tags'! (issue #280)
- it also supports wildcard namespaces, as now do regular namespace search predicates. both run faster. "crea*:anything" is now possible
- system:number of tags has been optimised, and in many cases is now ten to a hundred times faster
- system:number of tags still does not support siblings, something I hope to start correcting as of v408
- both tag existence (numtags =0 or greater than 0) and tag count database routines now respond quickly to 'cancel search' commands, so if you do run a slow query (a bare 'has creator tag' search on 'all known files' on the PTR, for instance), you can now back out quickly after the 'stop' button appears
- note that 'system:number of character tags greater than 0' and '= 0' are equivalent to +/-character:anything, which will be swapped in if you enter these. also, +/-unnamespaced:anything can now appear
- the program is a bit better about determining =0 and greater than 0 and less than 1 being 'none' and 'any but none', when it needs to determine optimisations and special labels
- unfortunately, I am taking away the default value for system:num tags in the options page (edit: I am killing the whole panel now). this old ugly mess of stacked predicate edit panels works on ancient, difficult to update code, so I will retire it and replace it with a unified system that is easy to use, supports in-search system predicate editing, and keeps up with changes automatically

Message too long. Click here to view full text.

Anonymous Board owner 07/22/2020 (Wed) 22:08:24 Id: b79965 [Preview] No.758 del
- database repair:
- the database menu has a new entry, 'repopulate truncated mappings tables', under the newly renamed 'check and repair' submenu, which will try its best to 'fix' a client.mappings.db file that has been truncated due to hard drive fault by repopulating from the local-file-only tag cache. do not run this unless you know you need to
- the 'help my db is broke.txt' document has a full update pass. the language is clearer, common issues and questions are better addressed, two new recovery routines are added, a section on the stages after boot recovery (like the new repopulate job above) is added, and I added my stock 'now become a backup patrician' nag at the end
- the debug routine to clear cached service info numbers is now moved to the 'regenerate' database menu. this thing fixes hanging incorrect 'pending' counts until I can fix it properly
- .
- the rest:
- fixed an issue where when you pasted queries into a subscription, those that were already in the sub (and got the dialog saying so), were being added anyway! I believe this bug came in the last few weeks, after the data storage rewrite. please check your pasted-into subs for dupes
- fixed tab double middle-click behaviour (so you can spam page close), which I thought I had fixed last week but actually messed up completely right at the end (issue #314)
- cleaned up some more of the page tab event code--it was a mess all around. should all be on Qt now, no wx hacks
- network jobs will no longer wait for and consume bandwidth start tokens while all network traffic is paused. all bandwidth competition now halts. (previously, they would continue to consume tokens according to current rules and then all rush to start as soon as traffic was resumed)
- fixed some client booru/client api requests to correctly 404 on missing file results, rather than 500
- cleaned up some file sort code and fixed the sort string conversion, which was rendering the opposite sort direction (asc/desc) in summary labels (e.g. on manage favourite searches)
- cleaned up some ui layout stretching code, including some borked tag import options expand sizing
- improved some button and padding layout definitions, and improved, slightly, the way the top-right media viewer hover window lays itself out and changes its size on media change
- improved some review services layout. should be fewer weird heights and widths in unusual situations, and the new multi-column list fits better
- the manage subs dialog now saves its changes to db more cleanly and atomically
- updated the default derpibooru parser to pull species tags. ten points if you can guess what that is most of the time

Message too long. Click here to view full text.

Release Tomorrow! Anonymous Board owner 07/28/2020 (Tue) 23:46:50 Id: 0286c0 [Preview] No.759 del
I had a good week. Along with some simpler code cleanup, I fixed some bugs, added some quality of life, and made it easy to detect and remove duplicate subscription queries, even across different subscriptions.

The release should be as normal tomorrow.

Version 404 Anonymous Board owner 07/15/2020 (Wed) 22:11:57 Id: d48e9d [Preview] No. 752 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=J84FPJAb7rM [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v404/Hydrus.Network.404.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v404/Hydrus.Network.404.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v404/Hydrus.Network.404.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v404/Hydrus.Network.404.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v404.tar.gz

I had a good week working almost entirely on the client's multi-column lists. They now remember their column widths and sort.

multi-column lists

All the tables across the program are now plugged into a unified options structure. This remembers the lists' columns' widths, so if you resize the manage subscriptions list a little tighter, then the next time you load that dialog, it should come back the same. The lists across the program that automatically start and maintain a sort (e.g. manage subs, or the query list in a downloader page), also remember if you change the sort.

Message too long. Click here to view full text.

Anonymous Board owner 07/15/2020 (Wed) 22:13:24 Id: d48e9d [Preview] No.753 del
full list

- column lists:
- all multi-column lists across the program now remember the widths of their columns when they are next recreated
- the last column of any list is now universally the 'stretching' column, which should correctly initialise with its preferred/previous size, but also grows and shrinks with the parent window
- while all lists retain their initial rows height, and those in the gallery and watcher management panels will continue to grow and shrink in a fixed way, all lists in dialog windows can now be shrunk down to four rows
- the minimum size of any column is now much smaller, about three characters
- all column headers now tooltip their name
- lists should be better about sizing in non-100% OS UI scalings
- the lists that are automatically sorted (e.g. the download pages, and manage subs) now remember the last sort you gave them
- future plans, now within reach:
- all lists will sort, sort arrows will appear on the header, and sort will be faster
- columns will be rearrangeable
- columns will be hide/showable, and initially hidden complicated columns will be available
- there will be some, maybe optional, capability to have lists sync live, so if you edit one, the others do the same
- num rows height memory, maybe--we'll see how the above shakes out first
- boring code changes:

Message too long. Click here to view full text.

Anonymous 07/16/2020 (Thu) 02:03:07 Id: e9c10a [Preview] No.754 del
holy shit, either this update or the last one fixed the way coloms work, now instead of working backwards from how you would think, they work correctly.


Release Tomorrow! Anonymous Board owner 07/21/2020 (Tue) 23:43:11 Id: 6d9527 [Preview] No.755 del
I had a good week. As well as misc cleanup and bug fixes, I added a namespace filter to 'system:number of tags', so you can now search for files with 2 series tags, more than four character tags, or any combination you can think of. Most 'number of tags' searches are faster, as well.

The release should be as normal tomorrow.

Version 403 Anonymous Board owner 07/08/2020 (Wed) 22:36:29 Id: e92248 [Preview] No. 744 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=iMzP0ZK-J7U [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v403/Hydrus.Network.403.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v403/Hydrus.Network.403.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v403/Hydrus.Network.403.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v403/Hydrus.Network.403.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v403.tar.gz

Despite losing a couple days to sickness, I had a great two weeks doing small jobs.


I reworked some of the behind the scenes of shortcut commands, most importantly cleaning up a bad decision I made when I first moved to the new system. There are about 120 'simple' commands in the system, and now all of them are defined in a neater way that allows me to change and expand things more easily in future.

Message too long. Click here to view full text.

1 post omitted.

Anonymous Board owner 07/08/2020 (Wed) 22:40:22 Id: e92248 [Preview] No.746 del
client api for advanced users

We need to experiment with this more, so feel free to play with this if you know what it is about, but for now this is mostly just for Client API developers, so we know what we need to do to get this working nicely.

The Client API (and the ancient and less-used local booru) now tentatively support https. You turn it on under services->manage services. It works like the hydrus server, and the same caveats apply: the certificates behind the https are 'self-signed', which means they work but are imperfect, and any program that wants to talk to them may have to access in a 'do-not-verify' way, or only after adding a specific exception rule, like you sometimes see in web browsers. If your Client API is actually hosted on a domain and you have a proper cert for it, feel free to replace the client.crt and client.key files that will be created in the db dir--the client should pick them up and use them no problem when it next boots.

full list

- shortcuts:
- shortcuts have a backend update this week. a bunch of hacky stuff is now cleaner behind the scenes, and the related UI has some cleanup as well
- converted all 100-odd simple shortcut commands from hacky text ids to a proper enumerated id system, and across every single instance across the program
- wrote nicer descriptive labels for all simple shortcuts. gone is 'focus_media_viewer', now is 'keyboard focus: to the media viewer'
- if you have no like/dislike or numerical services, the respective application command edit panels now say so and do not allow an ok action
- like/dislike rating sub-panels now start with 'like' checked
- when a like/dislike or numerical rating sub-panel is set to 'remove', the action dropdown is set to 'set' (rather than flip) and is disabled, as is the numerical slider
- application commands now state better "3/5" information about rating actions, rating than the underlying "0.6" float implementation
- all application commands existing in shortcuts or elsewhere are updated to the new enumerated id system

Message too long. Click here to view full text.

Anonymous Board owner 07/08/2020 (Wed) 22:40:50 Id: e92248 [Preview] No.747 del
- menu and UI cleanup:
- removed an old wx hack that prohibited last-second ui updates. the exit splash screen now reports final db shutdown info
- if a service or account is currently non-functional (e.g. all repositories are paused), the appropriate status text is now in red
- if there is work to do the first time a duplicate page is opened or looked at, it now moves to the 'preparation' tab
- doing a 'migrate database' file migration now temp-closes the migrate db dialog and hides the main gui while it goes on
- brushed up the tag filter ui a bit--now only one of the tag_filter/blacklist test phrases only show up, in the appropriate context, and the test text input now supports multiple newline-separated tags (e.g. if you want to paste a bunch)
- every panel on review services now has a refresh button to force an update
- the 'clear trash' button on the trash review services panel is now disabled when there is nothing to clear
- updated edit subscription panel to point to the main html help and brushed up that help to talk about file limits more, also the earlier downloader help has a little section to highlight subscriptions and their use
- reworked the 'restore from db backup' command--it is now integrated into client shutdown proper, and reports its basic restore progress to the exit splash screen
- reorganised the 'network' menu. manage subs is now up top, downloader submenus are now split better into high-level vs component-level, and login stuff is pulled to its own submenu
- put 'network traffic' at the top of the network->pause menu
- rearranged some of the 'gui' and 'gui pages' option pages and tucked everything into box sections for clarity
- the search pause/play button on search page tag autocomplete now has a simpler 'search paused' label when paused. the code has a similar nomenclature change, and eventually this will turn into a simple pause/play icon button or similar
- fixed some weirdness with floating autocomplete dropdowns sometimes not appearing on dialogs on first load
- fixed some focus logic so set-focus calls on downloader pages should work again on the query input text box and elsewhere
- unified all numerical rating->stars and stars->rating calculations across the program. this may have fixed some edge-case bugs

Message too long. Click here to view full text.

Anonymous Board owner 07/08/2020 (Wed) 22:41:27 Id: e92248 [Preview] No.748 del
- the local booru and client api now support the same https as the hydrus server, using self-signed certificates stored in the db directory. just set the checkbox in manage services and you should be good. self-signed certificates are free and will work on a server hosted off an IP address, but they are imperfect. they are also likely to require special permission to be accepted by the web browser or whatever you want to talk to the https service. however, if you host your client from a real DNS domain and have your own fully signed cert+key files, you can swap them in no problem
- local booru and client api urls adjust scheme for the new option, and unified and cleaned up how booru share urls are generated internally
- the way cert+key files are generated is moved from server code to common hydrus code
- cleaned up how additional db files like certificate files and the mpv conf are managed for backup/restore operations
- cleaned up some ancient http urls to https. mostly stuff like the regex tutorial links
- when files are appended to a regular search page (e.g. from a subscription publish to an existing page, or from a mouse drag and drop), the search context will now pause. this is to stop accidental F5 or mass refresh signals wiping out the changed page
- to break advanced-case gallery search loops, gallery url jobs now have a 'run' identity token. galleries pass their token down to 'next page' or 'sub-gallery' urls they generate, meaning all urls of a particular search run share the same url. gallery logs now ignore to-be-added urls that already exist for their token, terminating loops. new tokens are generated if a search is restarted or similar, meaning duplicate urls can exist in a gallery log, just not from the same starting point (issue #302)
- improved simple gallery url deduplication in several stages of the downloader pipeline
- when right-clicking on multiple thumbs, the info lines off the top menu item now list the files' combined viewtimes (this previously only showed when one file was selected)
- fixed some error reporting problems with adding urls to import via the client api--some url class exceptions were being converted from 400 to 500 errors unintentionally
- a new stylesheet, 'Hydracula', is added to the default install. check it out under options->style. thank you to a user for contributing this
- subscriptions are better about calculating a 90 second forgiveness window for bandwidth rules. they should schedule and startup more effectively, and the edit subscriptions and single edit subscription panels should also no longer show bandwidth delays below the next 90s, which are often a technical situation of regular work breaks that are better ignored for the purposes of the dialog
- went back up to pyinstaller 3.6 again on windows, as 3.5 caused its own Qt bindings dll problems. if you had trouble with 3.6 (401), let me know how this works for you, as there are additional dll-finding fixes included (issue #329)
- fixed an issue where under some conditions, file save dialogs were only happy with filenames that already existed (issue #319)

Anonymous Board owner 07/08/2020 (Wed) 22:42:03 Id: e92248 [Preview] No.749 del
- fixed an issue with the 'client already running' system sometimes not closing the client process correctly when told to cancel the boot
- bumped the 'space needed for vacuum' estimate up to 120% (was 100%) of estimated final file size, just to catch some edge cases
- rolling out updated danbooru parsers that pull associable urls correctly, thank you to a user for this fix
- rolling out an updated deviant art parser that finds some unusual file urls when other methods fail, thank you to another user for this fix (issue #295)
- upgraded cloudscraper to 1.2.42
- improved some type hinting
- fixed up some unit tests for new command and rating data

next week

Next week is a 'medium sized job' week. I want to write a 'list manager' to manage and remember presentation options for all the multi-column lists across the program. The ideal here will be to have selectable columns and columns that remember their last width. It will open the program to supporting clever and specific columns, initially hidden due to size constraints, for things like the edit subscriptions list in future. There are many lists in the program, and a lot of code here, so I am not sure how far I will get, but I think I'll be able to do column width memory at the least.

Simple Release Tomorrow! Anonymous Board owner 07/15/2020 (Wed) 03:42:24 Id: c27d3d [Preview] No.751 del
I had a good, full week that was almost entirely focused on migrating all the multi-column lists in the program to a unified options system. All lists in the program now remember their columns' widths, and those that typically sort now also restore their last sort.

This is a neat change, and it means it will be easier to add column selection and rearrangement in future, but that's basically it for the week. If you aren't interested in columns, feel free to skip!

The release should be as normal tomorrow.

Version 402 Anonymous Board owner 06/24/2020 (Wed) 22:06:32 Id: c46e45 [Preview] No. 727 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=e2VTFTFufHk [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v402/Hydrus.Network.402.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v402/Hydrus.Network.402.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v402/Hydrus.Network.402.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v402/Hydrus.Network.402.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v402.tar.gz

I had a great week just doing cleanup and other small work.

all misc this week

Most of the pages in the client now have special status bar text for when they have no files. If your search results in no files, instead of the old '0 files', you'll get 'no results for this search', and 'search cancelled!' if the search was cancelled, and so on. It just adds a bit of quick feedback to some actions--like highlighting a gallery query with no new files--where the new blank page may just be replacing another blank page.

Message too long. Click here to view full text.

7 posts and 2 images omitted.

Anonymous 07/04/2020 (Sat) 11:37:01 Id: f45698 [Preview] No.739 del
Yeah that option was ticked for some reason. Working now. Thanks and get better.

No Release Tomorrow! Anonymous Board owner 07/04/2020 (Sat) 21:47:15 Id: 61c373 [Preview] No.740 del
Thanks. Back in business now. Just knocked me out a couple days.

Anonymous 07/07/2020 (Tue) 12:21:23 Id: 8b5b1f [Preview] No.741 del
It's good that you're better now. Given that I'm pretty sure Hydrus has a bus-factor of 1 currently, It'd be very bad for everyone (including you, of course) if you were out of commission in any serious capacity. And also that would just suck on its own to hear that you bit the dust or got hospitalized or something bad like that.

Stay safe!

Release Tomorrow! Anonymous Board owner 07/08/2020 (Wed) 01:25:34 Id: 61c373 [Preview] No.742 del
I was knocked out for a couple days from illness, but I otherwise had a great couple of weeks doing small work. Shortcuts are nicer to work with, a bunch of UI is cleaned up and fixed, and tag import options now supports an explicit whitelist for requiring tags for file import.

The release should be as normal tomorrow.

Anonymous Board owner 07/08/2020 (Wed) 01:34:31 Id: 61c373 [Preview] No.743 del
Yeah, I don't really know what to do about the 'what happens if I get run over?' problem. This whole thing has lasted way longer than I thought it would, and I know it has lasted longer than the drama-bomb ragequit teamwork always induces in me, so mostly I am just grateful.

I know there are some programmer users who would might work together to keep something going, if they had the time and all that, but in a way, maybe 'last online 635d ago' is appropriate for an imageboard project, ha ha.

Version 401 Anonymous Board owner 06/17/2020 (Wed) 22:21:19 Id: 99c6a7 [Preview] No. 721 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=uHn9WnHe2eQ [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v401/Hydrus.Network.401.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v401/Hydrus.Network.401.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v401/Hydrus.Network.401.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v401/Hydrus.Network.401.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v401.tar.gz

I had an ok week, although I was a little short on work time. This is mostly a bugfix release that tweaks how the new subscriptions work and fixes some downloader update issues. There is also an important new way to make feature suggestions for hydrus.

mostly misc work this week

Now that subscriptions can run in small efficient pieces, I removed several old limiters that stopped the old system running too often. Subscriptions should boot with less delay when needed (e.g. after the client loads or when you ok the dialog after changes), and an old system that batched queries together is gone, so if a query is due, it should run generally as soon as it can.

Message too long. Click here to view full text.

Anonymous Board owner 06/17/2020 (Wed) 22:23:00 Id: 99c6a7 [Preview] No.722 del
github issue tracker

As you doubtless know, I have difficulty keeping up with messages. It is great that people like hydrus and think of all sorts of ways to make it better, but as the program has become more popular, feature suggestions in particular has grown into a tsunami that I cannot give the time they deserve. My todo list has about 3,000 items in it. My list is private, so dealing with duplicate requests and keeping track of discussions in threads and chat conversations has been inefficient.

So, with the aid of a team of volunteer users, I am turning the Hydrus Github Issue Tracker back on. I turned it off about three years ago, as it never fit my workflow and I could not keep up with discussions, but I have been talking with some users, and I agree that for the people it does work for, it is excellent.

We are still figuring out the workflow, but the idea is the team of users will label and sort things, participate in discussions about features, and forward to me the most urgent and popular items. I will not be actively managing it, but I'll read the issues I am forwarded and likely close things out once I have them done. If you are comfortable with Github, please feel free to participate yourself. If you have already requested a feature to me, you can still repeat it on the tracker, which will help deduplication for other users and provide a clean space to discuss it with others if you would like. The program and the html help has all been updated with info about and links to the tracker. It is on the main repo, here:


I am grateful for the volunteers who are taking on a job my mind just doesn't gel with.

You are still welcome to contact me on any subject however else we used to talk. This is simply a new way to report, particularly for feature suggestions. If you are uncomfortable with Github, no worries. Also, if you would like me to forward an item to the Tracker anonymously, please send me it and I will pass it along.

Anonymous Board owner 06/17/2020 (Wed) 22:24:16 Id: 99c6a7 [Preview] No.723 del
full list

- subscriptions:
- as subs can now load more flexibly, previously hardcoded waits are now eliminated:
- - the subscriptions manager now only waits three seconds after initial session load to boot (previously 15)
- - the subscriptions manager now wakes as soon as the subscriptions dialog is ok'd or cancelled
- - a timing calculation that would delay the work of a sub up to five or fifteen minutes if more queries would come due for sync in that time window (in order previously to batch to reduce read/write) is now eliminated--subs will now start as soon as any query is due. if you were ever confused why a query that seemed due did not boot after dialog ok or other wake-up event, this _should_ no longer happen
- re-added the import/export/duplicate buttons to manage subs. export and dupe may need to do db work for a couple of seconds and will have a yes/no confirmation on larger jobs
- the import button on manage subs accepts and converts the old 'legacy' subscription object format, including a copy/paste of the objects backed up to disk in the v400 update
- fixed an issue where creating a subscription query and then deleting it in the same manage subs dialog session would result in surplus data being written to the db (which the next dialog launch would note and clear out)
- an unusual error with pre-run domain checking, exposed by the new subscription code and e621 subs, where the gallery url has also recently changed, is now fixed
- .
- issue tracker:
- the Github issue tracker (https://github.com/hydrusnetwork/hydrus/issues) is turned on again! it is now run by a team of volunteer users. the idea is going to be to try to merge duplicate feature suggestions with the proper platform and put some discussion and cognition and prioritisation into idea development before it gets to my desk, so I can be more focused and productive and so 95% of feature suggestions do not simply get banished to the shadow realm of the back of my todo
- this is mostly intended for wishlist and other suggestions, as the tsunami was just getting too much for me to handle, but we'll see how it goes for things like bug reports as well. I'll still take any sort of report through my normal channels, if you are uncomfortable with github, or if you wish for me to forward an item to the issue tracker anonymously
- the website, help documents, and hydrus help menu links have been updated regarding the issue tracker

Anonymous Board owner 06/17/2020 (Wed) 22:24:44 Id: 99c6a7 [Preview] No.724 del
- the rest:
- improved how the database 'update default downloader objects' job works, ensuring that new defaults are better at simply take the place of existing objects and do not break/reset existing url class to parser links
- tightened up how automatic url class to parser linking works, eliminating some surplus and potentially bad data related to api links. furthermore, whenever the links between url classes and parsers update, existing surplus data, which may creep in when api links change, is now cleaned from the data structure
- rolling out updated e621 url class and parser to deal with their alternate gallery url format
- rolling out an updated derpibooru parser that will link to the new api class correctly
- thanks to a user's submission, rolling out updated versions of the new default nitter parsers that pull creator:username tags
- before every subprocess launch, and when waiting for all subprocess communication (e.g. to ffmpeg), now tests regularly for program shutdown. if an unusual situation develops where a subscription is doing a file import job while the OS is shutting down, and that system shut down would hang or is hanging on a 'ffmpeg can't be launched now' dialog, the hydrus client should now notice this and bomb out, rather than going for that never-running ffmpeg. this may not fix all instances of this issue, and further feedback on the client not closing down cleanly with the OS is welcome.
- when adding a new path to the 'migrate database' panel, any symbolic links will be converted to canonical equivalents
- added some location checks and appropriate errors when the database is doing file storage rebalancing
- fixed an issue uploading swfs, video, or audio to the server when it is launched from a frozen executable build
- misc code cleanup

next week

Next week is cleanup. I want to untangle some of the convoluted shutdown code, push more on wx->Qt cleanup, and revisit the new text parsing so I can get closer to multiline note parsing for the downloader.

Release Tomorrow! Anonymous Board owner 06/24/2020 (Wed) 05:56:30 Id: 26036f [Preview] No.726 del
I had a great week doing small work. I cleaned a heap of old code, fixed some bugs, and added some quality of life.

The release should be as normal tomorrow.

github issues Anonymous 06/13/2020 (Sat) 22:30:11 Id: 6b2d8f [Preview] No. 719 [Reply] [Last 50 Posts]
It looks like Hydrusdev enabled "issues" on the github repo a few hours ago. Was there any update or notification given by him about this? He seemed pretty insistent on using the boards and his own personal to-do list for keeping track of development and bugs, so I'm curious if he explained somewhere why he changed his mind.

Anonymous Board owner 06/24/2020 (Wed) 05:05:51 Id: f3e2b9 [Preview] No.725 del
Sure, check the latest release post >>722.

tl;dr: A team of users are managing it, they'll pass me work regularly, you can still talk to me any other way you did before.

Version 400 Anonymous Board owner 06/11/2020 (Thu) 14:30:06 Id: a73bbc [Preview] No. 715 [Reply] [Last 50 Posts]

Message too long. Click here to view full text.

Anonymous Board owner 06/11/2020 (Thu) 14:30:28 Id: a73bbc [Preview] No.716 del
full list

- subscription data overhaul:
- the formerly monolithic subscription object is finally broken up into smaller pieces, reducing work and load lag and total db read/write for all actions
- subscriptions work the same as before, no user input is required. they just work better now™
- depending on the size and number of your subscriptions, the db update may take a minute or two this week. a backup of your old subscription objects will be created in your db directory, under a new 'legacy_subscriptions_backup' subdirectory
- the manage subscriptions dialog should now open within a second (assuming subs are not currently running). it should save just as fast, only with a little lag if you decide to make significant changes or go into many queries' logs, which are now fetched on demand inside the dialog
- when subscriptions run, they similarly only have to load the query they are currently working on. boot lag is now almost nothing, and total drive read/write data for a typical sub run is massively reduced
- the 'total files in a sub' limits no longer apply. you can have a sub with a thousand queries and half a million urls if you like
- basic subscription data is now held in memory at all times, opening up future fast access such as client api and general UI editing of subs. more work will happen here in coming weeks
- if due to hard drive fault or other unusual situations some subscription file/gallery log data is missing from the db, a running sub will note this, pause the sub, and provide a popup error for the user. the manage subscription dialog will correct it on launch by resetting the affected queries with new empty data
- similarly, if you launch the manage subs dialog and there is orphaned file/gallery log data in the db, this will be noticed, with the surplus data then backed up to the database directory and deleted from the database proper
- subscription queries can now handle domain and bandwidth tests for downloaders that host files/posts on a different domain to the gallery search step
- if subs are running when manage subs is booted, long delays while waiting for them to pause are less likely
- some subscription 'should run?' tests are improved for odd situations such as subs that have no queries or all DEAD queries
- improved some error handling in merge/separate code
- the 'show/copy quality info' buttons now work off the main thread, disabling the sub edit dialog while they work

Message too long. Click here to view full text.

Anonymous Board owner 06/11/2020 (Thu) 14:30:58 Id: a73bbc [Preview] No.717 del
- client api:
- the client api now delivers 'is_inbox', 'is_local', 'is_trashed' for 'GET /get_files/file_metadata'
- the client api's Access-Control-Allow-Headers CORS header is now '*', allowing all
- client api version is now 12
- .
- downloaders:
- twitter retired their old api on the 1st of June, and there is unfortunately no good hydrus solution for the new one. however thanks to a user's efforts, a nice new parser for nitter, a twitter wrapper, is added in today's update. please play with it--it has three downloaders, one for a user's media, one for retweets, and one for both together--and adjust your twitter subscriptions to use the new downloader as needed. the twitter downloader is no longer included for new hydrus users
- thanks to a user's submission, fixed the md5 hash fetching for default danbooru parsers
- derpibooru gallery searching _should_ be fixed to use their current api
- .
- the rest:
- when the client exits or gets a 'modal' maintenance popup window, all currently playing media windows will now pause
- regrettably, due to some content merging issues that are too complicated to improve at the moment, the dupe filter will no longer show the files of processed pairs in the duplicate filter more than once per batch. you won't get a series of AB, AC, AD any more. this will return in future
- the weird bug where double-clicking the topmost recent tags suggestion would actually remove the top two items _should_ be fixed. general selection-setting on this column should also be improved
- middle-clicking on a parent tag in a 'write' autocomplete dropdown no longer launches a page with that invalid parent 'label' tag included--it just does the base tag. the same is true of label tags (such as 'loading...') and namespace tags
- when changing 'expand parents on autocomplete' in the cog button on manage tags, the respective autocomplete now changes whether it displays parents
- this is slightly complicated: a tag 'write' context (like manage tags) now presents its autocomplete tags (filtering, siblings, parents) according to the tag service of the parent panel, not the current tag service of the autocomplete. so, if you are on 'my tags' panel and switch to 'all known tags' for the a/c, you will no longer get 'all known tags' siblings and parents and so on presented if 'my tags' is not set to take them. this was sometimes causing confusion when a list showed a parent but the underlying panel did not add it on tag entry

Message too long. Click here to view full text.

Anonymous Board owner 06/11/2020 (Thu) 14:32:35 Id: a73bbc [Preview] No.718 del
- the '... elide page tab names' option now applies instantly on options dialog ok to all pages
- added 'copy_bmp_or_file_if_not_bmpable' shortcut command to media set. it tries copy_bmp first, then copy_file if not a static image
- fixed some edit tag filter layout to stop long intro messages making it super wide
- fixed an issue where tag filters could accept non-whitespace-stripped entries and entries with uppercase characters
- fixed a display typo where the 'clear orphan files' maintenance job, when set to delete orphans, was accidentally reporting (total number of thumbnails)/(number of files to delete) text in the file delete step instead of the correct (num_done/num_to_do)
- clarified the 'reset repository' commands in review services
- when launching an external program, the child process's environment's PATH is reset to what it was at hydrus boot (removing hydrus base dir)
- when launching an external program from the frozen build, if some Qt/SSL specific PATH variables have been set to hydrus subdirectories by pyinstaller or otherwise, they are now removed. (this hopefully fixes issues launching some Qt programs as external file launchers)
- added a separate requirements.txt for python 3.8, which can't handle PySide2 5.13.0
- updated help->about to deal better with missing mpv
- updated windows mpv to 2020-05-31 build, api version is now 1.108
- updated windows sqlite to 3.32.2

next week

Next week is a small jobs week. I'll get subscription import/export working and then hack away at my bugs list and other little things to catch up on.

Simple Release Tomorrow! Anonymous Board owner 06/17/2020 (Wed) 00:25:03 Id: 7b2e5c [Preview] No.720 del
I had an ok week, though a little short on work time. I tweaked how subs work for the new system, getting them booting a little faster, re-added import/export/duplicate for subs (including importing+conversion from the old format), and cleaned up some bugs, including recent problems with linking certain downloader components together.

The release should be as normal tomorrow.

Version 399 Anonymous Board owner 05/27/2020 (Wed) 22:19:07 Id: 15fb94 [Preview] No. 700 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=i_u3hpYMySk [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v399/Hydrus.Network.399.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v399/Hydrus.Network.399.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v399/Hydrus.Network.399.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v399/Hydrus.Network.399.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v399.tar.gz

I had a great week tidying up smaller issues before my vacation.

all small items this week

You can now clear a file's 'viewing stats' back to zero from their right-click menus. I expect to add an edit panel here in future. Also, I fixed an issue where duplicate filters were still counting viewing time even when set in the options not to.

Message too long. Click here to view full text.

5 posts omitted.

Anonymous Board owner 06/03/2020 (Wed) 20:00:34 Id: d41e66 [Preview] No.708 del
Ah, sorry, I think that setting requires a restart to take effect for existing pages. If I can't make it work immediately, I'll make sure to add a text note about it in the options.

Anonymous 06/03/2020 (Wed) 22:28:06 Id: efc5cb [Preview] No.711 del

Ah, yeah. Restart helped. Nice.

Thank you!

Anonymous 06/07/2020 (Sun) 18:27:33 Id: d97ce3 [Preview] No.712 del
A while back audio would skip on some files.
I don't know when, but for me the problem is fixed, I just started to go though some of the tabs I have open and found the 'skipping music sound' buried.

turns out they all work now.

Anonymous Board owner 06/10/2020 (Wed) 05:30:25 Id: d41e66 [Preview] No.713 del
Hey, cool. I cannot take any credit though. I have been updating the mpv that hydrus uses, and there will be another one with v400, so perhaps it got fixed on their end.

Release on Thursday! Anonymous Board owner 06/10/2020 (Wed) 05:30:52 Id: d41e66 [Preview] No.714 del
I had a great vacation, and now a great week doing an important and long-planned subscriptions rewrite that will make subscriptions load and operate much faster and use less memory.

This is a big change, one that has monopolised my week, so I will take Wednesday to do more testing on my IRL client and also some other work like bug fixes.

The release should on Thursday, hopefully in the morning USA time.

Version 398 Anonymous Board owner 05/20/2020 (Wed) 22:23:28 Id: ddb1ab [Preview] No. 689 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=agfhkR6JYEQ [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v398.tar.gz

I had a good work week. Tag autocomplete gets some new search options, and advanced users who make downloaders get some new text processing tools.

tag autocomplete

When I recently overhauled the tag autocomplete pipeline, I eliminated some unusual logical hoops where you could accidentally fire off expensive searches that would fetch all tags. Now the code is clean, I am adding them back in as real options.

Message too long. Click here to view full text.

6 posts omitted.

Anonymous 05/25/2020 (Mon) 21:04:16 Id: ede36e [Preview] No.696 del
(55.41 KB 909x614 error.png)
Whenever I try to change any GUG, url class, or parser key element on thi version, I get this popup. Example text for the context doesn't matter, the only key elements that aren't affected are elements that accept all characters.

Anonymous 05/26/2020 (Tue) 03:49:37 Id: ede36e [Preview] No.697 del
In order to change fixed character/characterset keys to different values without that error appearing,

1. regex should be selected
2. example text changed to match intended key value
3. appropriate key type should be selected.

The example text field isn't visible outside of the regex key definition, so it cannot be changed from the default example or previous example it complains about.

Anonymous 05/26/2020 (Tue) 10:15:47 Id: 143c00 [Preview] No.698 del
found something interesting.

I am very use zoom just zooming the center, still not use to it zooming to where I point.

Now with duplicates, i'm using a mouse, but when im just culling general files, i'm using a g13,so the mouse is, somewhere, on screen. Now in some very side image, my mouse has been off image, the one i'm on now that made me realize what was happening is 3000x1000.

so i'm at the bottom of the window, I hit the zoom 100 to read text, and the image is now nearly off screen because it zoomed into the bottom of the screen.

so begins a 'problem'

would it be possible to have 2 different zoom methods? 1 for when a click is how you decided something and one for when a click is more likely to just move the image around? honestly, since I know control cancels out the click in duplicates i'm thinking of using a different zoom because zooming right to mouse is, while still useful, less useful then it was.

Release Tomorrow! Anonymous Board owner 05/27/2020 (Wed) 00:25:20 Id: 24f2fd [Preview] No.699 del
I had a great week just fixing bugs and making small improvements.

The release should be as normal tomorrow.

Sorry, this is fixed in tomorrow's release! When I hid and stopped overwriting the 'example text' input for 'fixed' type, I forgot to update the 'ok' validity test.

Anonymous Board owner 05/30/2020 (Sat) 20:18:17 Id: 94cbbd [Preview] No.703 del
Yeah, I think this is the best solution. I'll make some new zoom shortcuts that override the zoom center.

Version 397 Anonymous Board owner 05/13/2020 (Wed) 20:20:19 Id: 3f1070 [Preview] No. 676 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=2kEC7X1FUIg [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v397/Hydrus.Network.397.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v397/Hydrus.Network.397.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v397/Hydrus.Network.397.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v397/Hydrus.Network.397.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v397.tar.gz

I had an ok week. I mostly did small fixes and improvements.


I added 'system:has/has no note with name xxx' to search for files with specific note names. This is bundled into the same dialog as last week's 'number of notes', for the new generic 'system:notes' entry in the normal system predicates list.

Message too long. Click here to view full text.

4 posts omitted.

Anonymous 05/18/2020 (Mon) 20:52:42 Id: ff3815 [Preview] No.684 del
(13.63 KB 411x490 client.png)
I have the "Hide the preview window" setting turned on, but since v397 the preview window remains visible as a black box. It doesn't load any actual previews, but previously the window wouldn't be there at all
Is it possible to hide the black box again? Resizing the left pane down to hide it only does so for the current session; it returns when Hydrus restarts

Anonymous 05/20/2020 (Wed) 00:56:13 Id: dfba89 [Preview] No.685 del
I just started using Hydrus. As a trial, I gave it 1000 files (about 700MB) to import and tag. The concerning part is that the db folder is 18GB in size and contains 28000 files. This represents ~25x increase in size after import. Is this going to continue linearly for the rest of my library or is the delta a one time increase in library size? If it's the former, did I do something wrong to increase the size of the db by accident?

Release Tomorrow! Anonymous Board owner 05/20/2020 (Wed) 04:17:52 Id: 3250df [Preview] No.686 del
I had a good work week. I added some tag autocomplete search options, so you can now enable '*', 'namespace:*', and '"ser" matches "ser*:*"' wildcard searches for particular services, expanded the text processing tools for the users who make downloaders, and also did a little quality of life.

The release should be as normal tomorrow.

Anonymous Board owner 05/20/2020 (Wed) 04:26:50 Id: 3250df [Preview] No.687 del
Did you sync with the Public Tag Repository? Or maybe download a pre-synced database?

A typical client stores the media file and a thumbnail, but this usually only adds up to about 102% of the total size. The PTR however has a huge number of tags, hundreds of millions, built up over years of user submissions that is processed into the database files proper, and comes with a few gigabytes of update files. I would estimate a fully PTR synced client is about +25GB or so. Once you are fully synced with it, it grows only a bit every day as new tags come in.

If you check your install_dir/db folder, there should be four client..db files. Are those eating most of the space? Inside your client_files directory should be your actual files--they should total about 2,000 files for your media and thumbnails and perhaps 8,000 update files. I am not sure where the ~18,000 extra files are coming from, unless you imported 10,000 rather than 1,000.

If you are interested, there is a bit more on the db structure here:


Let me know if you are not synced with the PTR. If you have a different program accessing that directory, could it possibly be creating thumbnail caches in every client_files subdirectory or something?

Anonymous Board owner 05/20/2020 (Wed) 04:28:09 Id: 3250df [Preview] No.688 del
Thanks, I will check this tomorrow morning. The new flexible layout code must not be obeying that setting correctly.