Catalog of /hydrus/

Bottom

Mode: Thread

Max file size: 350.00 MB

Max files: 5

Remember to follow the rules

Max message length: 4096

Open

R: 0 / I: 0 / P: 1

Version 597

https://youtube.com/watch?v=Q8Jbel4M4Uc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v597/Hydrus.Network.597.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v597/Hydrus.Network.597.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v597/Hydrus.Network.597.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v597/Hydrus.Network.597.-.Linux.-.Executable.tar.zst

I had a good week. There's some more bug fixes and improvements to quality of life.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

When I sped up subscription and downloader load time a couple weeks ago, I messed something up with local import folders that caused them to one-time re-check of any files they were remembering (i.e. entries where you said 'leave alone, do not reattempt'). It resulted in some wasted work and an UI bug. Thank you for the reports about this. I have fixed the problem, and any duplicate/redundant import objects should be removed on update. Let me know if you have any more trouble!

I worked a bit more on copying tag parents, which wasn't has helpful IRL as I expected. I've moved the default 'copy' behaviour (e.g. when you hit Ctrl+C) back to the old 'just copy without parents', and now the 'copy' menu has a new option for specifically copying with parents if there are any. I also extended support for this to more taglist types and removed some accidental parent-indent garbage in the copy code.

Mr Bones, the export files window, and the manage times, urls, and notes dialogs now all have 'frame location' entries under options->gui. If you always want 'edit notes' to appear on a second monitor or something, you can now set it up.

If you are an advanced user that does parsing stuff, all formulae now have an option and purely descriptive 'name/description' field. Feel free to start naming the more obscure parts of ZIPPERs or whatever you are working with. Also, the 'edit ZIPPER' panel, the main 'edit formula' panel (where you can change formula type), and the 'edit string processor' panel all now have import/export/duplicate buttons! Should be a bit easier to copy complicated regex and stuff around.

Win 7

I am not totally sure, but it looks like we lost Win 7 support back in v582 or so. Some of the libraries we use are getting trickier to build on Win 7, and some newer hydrus code simply will not run on Python 3.8, which is the latest Windows 7 can run. I have updated the 'running from source' help to talk specifically about this.

We knew this train was coming, and it looks like it is suddenly here. Windows 7 users are stuck on source ~v582 until they update Windows or move to Linux/macOS. Thanks for using hydrus!

future build

Only for advanced users!

I am making another future build this week, but just for Windows. This is a special build with libraries that I would like advanced users to test out so I know they are safe to fold into the normal release. It is newer SQLite and mpv dlls this time. More info in the post here:

https://github.com/hydrusnetwork/hydrus/releases/tag/v597-future-1

next week

I have six weeks of work left before Christmas. I am not confident I will get duplicates auto-resolution working in time, but it'd be nice, and I'm having a great time doing code cleanup and widget overhaul along with it, so I'll keep on trucking like this until the end of the year, I think. So, more small jobs and overall cleanup.
Open

R: 7 / I: 2 / P: 1

Version 596

https://youtube.com/watch?v=amKt8ttja1E [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v596/Hydrus.Network.596.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v596/Hydrus.Network.596.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v596/Hydrus.Network.596.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v596/Hydrus.Network.596.-.Linux.-.Executable.tar.zst

I had a good week with a couple important bug fixes and some UI quality of life improvements.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I messed something up last week and it broke several downloaders' hash lookups, which are sometimes used to determine 'already in db'/'previously deleted' quickly. The problem is fixed today, sorry for the trouble! Any advanced users who took advantage of the new hex/base64 string converter decoding tech last week, please check the changelog.

I also fixed an issue with the Client API, where it was not dealing with file_ids that do not exist properly. If you were doing manual API jobs and put in a random id as a test and had any problems afterwards, let me know and we'll figure out the fix for your case.

The big ugly list of frame location info under options->gui has some new buttons to quickly flip 'remember size/position' and clear the 'last size/position' to several rows at once. I'm going to try to expand these listings to cover more window types--e.g. Mr Bones doesn't have one yet--so let me know if there is anything you would like to have its own size and position and stuff, and the whole thing, if I can get myself in gear, could do with a usability pass.

The review services panel will now not be so tall if you have the PTR. I hacked in some expand/collapse tech for a layout box I use all over the place. Let me know what you think, because I'll probably use it in some other places and figure out collapse memory and stuff.

Animations with an fps below 10 or 1 will now show to two significant figures, rather than just being rounded to the nearest integer. You'll see 1.2fps and 0.50 fps.

Thanks to a user, the Client API can now render Ugoiras into apng or animated webp!

next week

I did some good cleanup this week and moved duplicates auto-resolution just a little bit further forward. I'll keep pushing on that and do more small jobs.
Open

R: 1 / I: 0 / P: 1

Version 595

https://youtube.com/watch?v=2fsMD9tPFY0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v595/Hydrus.Network.595.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v595/Hydrus.Network.595.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v595/Hydrus.Network.595.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v595/Hydrus.Network.595.-.Linux.-.Executable.tar.zst

I had a great week. I've got several quality of life improvements, and a user has figured out native Ugoira rendering.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

Ugoiras

Ugoiras are an unusual gif-like animation format from Pixiv. Getting them to play in hydrus has been a long time goal, and thanks to a user who put a bunch of work in, they now work out of the box. You don't have to do anything; any Ugoiras you have in your client will now play in the media viewer.

If your Ugoiras have animation timing data inside their archive (those downloaded with PixivUtil or gallery-dl should have these!), hydrus will parse this and render frames for the correct duration. There is also an option to present this info in a file note. If there is no timing data, hydrus will fall back to 8fps.

There's more technical data here: https://hydrusnetwork.github.io/hydrus/filetypes.html#ugoira

All your Ugoiras will get a metadata rescan on update, so it might take a few minutes for any duration data to fully kick in.

We are now planning possible ways we can add the Ugoira timing data using the hydrus downloader engine, which would finally allow in-client downloading of properly animating Ugoiras. Let me know how today's update works for you, and we'll see what we can do.

other highlights

All multi-column lists now sort case-insensitively. A subscription called 'Tents' will now slot between 'sandwiches' and 'umbrellas'.

All 'file logs' in importers now load much faster. If you had slow subscription or session load, let me know how things feel now.

The 'Favourite Searches' star button now supports nested folders in a hacky way. Just put '/' in the 'folder name', and it'll make submenus for you.

I fixed a bug where setting a pair of potential duplicates as 'not related' would sometimes merge modified file dates. You might get a popup about it on update about it to reset file modified dates for any files that were previously affected.

OR predicates get some more work--you can now 'start an OR with selected', which opens the edit panel and then replaces, 'dissolve' an OR back to its constituent sub-preds, and the menu and UI got a little polish.

The 'urls' thumbnail right-click menu has a new 'force metadata refretch' command, which makes it easy to redownload Post URLs to get tags and stuff again. You don't have to manually set up the url downloader with a custom 'tag import options' any more--hydrus can do it for you.

new build

The build today folds in some library updates the advanced users and I have been testing. As far as we can tell, there are no special update instructions needed, so just update as normal. If you are on an old version of Windows/Linux and you have any problems booting, let me know.

next week

I did some placeholder duplicates auto-resolution UI this week, and I'm feeling better and better about the whole system. There's still a lot to do, but none of it looks too crazy. I'd like to keep pushing on it a bit. Otherwise I'd like to just do some boring code cleanup.
Open

R: 1 / I: 0 / P: 1

Version 594

https://youtube.com/watch?v=X6_e7v5Fe6Y [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v594/Hydrus.Network.594.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v594/Hydrus.Network.594.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v594/Hydrus.Network.594.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v594/Hydrus.Network.594.-.Linux.-.Executable.tar.zst

I had a good simple week fixing some bugs. There is also a 'future test' for advanced users to try out.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I screwed up last week when I updated how the list that backs the thumbnail grid works. I messed up an optimisation in how it removed certain items, and it caused people popup errors when deleting/removing files. The good news is these errors were just harmless UI side stuff, fixed by re-sorting the page or restarting the client, but I am sorry for the trouble--it is now fixed, and I have some unit tests to make sure it does not happen again.

The tag right-click->search menu, if you select stuff that is in the current search, now lets you say 'remove this selection and replace it with an OR of them'!

I cleaned some of the shutdown code. The program should exit a bit faster and smoother, particularly when it receives emergency signals to shut down real quick. If you are in an odd situation that is often sending SIGTERM or something, let me know how it goes.

Since it was tucked away before, I wrote out a fresh version of my 'two rules to not going crazy when tagging' here: https://hydrusnetwork.github.io/hydrus/getting_started_more_tags.html#tags_are_for_searching_not_describing

Qt and AUR

There's a new version of Qt, 6.8, rolling out this past week. AUR users had some trouble when they were auto-updated to it. A PySide6 hotfix came out yesterday to fix something, and I think I have fixed something else that users reported about the tab-bar, but I expect there will be more issues, so let me know what you run into. I'm still testing 6.7 on my own personal situation.

If you are just a normal Arch user, I now generally recommend not using AUR packages for python programs like hydrus because AUR stuff will always use the latest version of any python libraries. If there's a bug in the new Qt or numpy deprecates a particular call, you are then dealing with that. If you would prefer a more reliable solution that takes a couple more steps, please try running hydrus from source yourself: https://hydrusnetwork.github.io/hydrus/running_from_source.html

future test

Only for advanced users!

I am making another future build this week. This is a special build with libraries that I would like advanced users to test out so I know they are safe to fold into the normal release. More info in the post here:

https://github.com/hydrusnetwork/hydrus/releases/tag/v594-future-1

next week

I want to hammer more at duplicate auto-resolution. Maybe get an empty UI panel going.
Open

R: 1 / I: 0 / P: 1

Version 593

https://youtube.com/watch?v=OPDSEcIbeEE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v593/Hydrus.Network.593.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v593/Hydrus.Network.593.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v593/Hydrus.Network.593.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v593/Hydrus.Network.593.-.Linux.-.Executable.tar.zst

I had a good week. There's some fixes and tag autocomplete quality of life.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

OR search predicates are now multiline, one sub-predicate per line. They look a lot better! They also have better 'copy text' support--they'll refer to and copy all the sub-predicates separately, and do subtag conversions correctly.

Search pages' tag autocompletes will now better recognise if you type in a 'worse' sibling exactly. If you have 'lotr' siblinged to 'series:lord of the rings', typing 'lotr' will now promote the 'series:lord of the rings' tag to the top of the results, regardless of count.

You might get a yes/no dialog on update regarding your Client API permissions. I screwed up the 'should I do it?' test in the 'permit everything' permissions update a couple weeks ago, so if you were affected, it will ask if you want to re-do the update with fixed logic.

I fixed the 'source time' parsing for gelbooru and a bunch of gelbooru-engine downloaders. I'm not sure if something changed on their end or ours, but it should work better now.

I had success working on the duplicate auto-resolution database module this week, making a skeleton and then fleshing it out a bit. I feel really good about it. I estimate it to be 25-33% complete while the object work I previously did is 50-75% and the UI, not yet started, 0%. So, there's a good bit to go before we are 1.0 here, but all the obvious technical questions are answered and I see the path now.

next week

I want to focus on my github issues bug reports.
Open

R: 1 / I: 0 / P: 1

Version 592

https://youtube.com/watch?v=ldrcOdS8oAU [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v592/Hydrus.Network.592.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v592/Hydrus.Network.592.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v592/Hydrus.Network.592.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v592/Hydrus.Network.592.-.Linux.-.Executable.tar.zst

I had a great week. There's a mix of all sorts of work.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

The tag autocomplete widgets on search pages now have a one-click 'clear page' button. Also, tag autocompletes will handle ctrl+c/ctrl+insert a little more intelligently, copying your typed text if any is selected and otherwise copying from the results list below.

Last week's tags->migrate tags pair filtering in gets another checkbox, for 'A or B'.

The parsing system has a new formula type, NESTED, which holds two sub-formulae, passing the results from the first to the second. If you need to parse some HTML inside some JSON, or vice versa, I hope it is now easy! I renamed COMPOUND formulae to ZIPPER and updated a little of the help around here. String Processors also now all have copy/paste buttons, for easy migration.

The Client API gets another 'local path' command to fetch all the local file storage locations. If you need to hit up the whole filesystem for mass conversion or duplicate checks or whatever, you can now do it without the overhead of the individual file fetches.

I updated some library versions in the build and advanced venv scripts. If you run from source, I believe the program will no longer run on Python 3.7 (although I wouldn't be surprised if that were already true). As I understand, Win 7 can only run Python 3.8 at the latest, so we may be running out of track on that front. If you are an advanced source user and want to help me with a simple 'future' test, please check out the changelog.

next week

I didn't get to working on duplicate auto-resolution, so I'll try that again.
Open

R: 1 / I: 0 / P: 1

Version 591

https://youtube.com/watch?v=1ySAkB-H33E [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v591/Hydrus.Network.591.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v591/Hydrus.Network.591.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v591/Hydrus.Network.591.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v591/Hydrus.Network.591.-.Linux.-.Executable.tar.zst

I had a bit of a mixed week, but I got a couple of neat things done.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I fixed a stupid mistake in last week's 'move left/right after page close' thing, which was doing it even if the page being closed was not the current focus! Also fixed is the share->export menu not showing if you only right-clicked one file.

The tags->migrate tags dialog can now filter parent and sibling tags based on tag count. You can say 'only include pairs where the A/B actually exists on service x', where for siblings that B is the ideal tag at the end of the chain. I hope this makes it easier to filter giant multi-hundred-thousand pools of pairs, like from the PTR, down to only what matters for your 'my tags' etc..

The Client API has a new simple permissions mode for access keys, a convenience state just called 'permits everything', which does exactly what you think and is intended to be an easy catch-all for people who just want to turn on future-proofed access for an app they trust. On update, any access key that looks like it was told to permit everything in the old system will be updated to this mode (you will get a popup saying this happened, too). If you do need finer control, you can still tweak individual Client API permissions under services->review services.

Relatedly, the Client API gets a new permission this week, 'see local files', and commands to fetch local file and thumbnail paths. If you want to access files directly on the local machine for metadata analysis or whatever, this is what you want!

next week

I'm sorry to say I have been unproductive recently, so I'll keep it simple. I think I want to try some duplicate auto-resolution database stuff. I still need to think about a couple things, so I'll play around a bit and see what makes the most sense. Fingers crossed, I'll have the skeleton of the maintenance daemon sketched out.

R: 27 / I: 3 / P: 1

Suggestion Threads

Drop your suggestions here
http://archive.fo/heSey
Open

R: 1 / I: 0 / P: 1

Version 590

https://youtube.com/watch?v=eYN9WivpQ6M [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v590/Hydrus.Network.590.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v590/Hydrus.Network.590.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v590/Hydrus.Network.590.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v590/Hydrus.Network.590.-.Linux.-.Executable.tar.zst

I had a pretty good week and have a bunch of quality of life improvements to roll out.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

The 'check now' button in 'edit subscriptions' and 'edit subscription' is now more intelligent and handles pause status. It'll ask you about paused subscriptions and queries, and it better examines and navigates what you can and will be resurrecting.

If you shrink the search page's 'preview' window down to 0 pixel size, it now recognises it is hidden and will no longer load videos and stuff in the background!

In the parsing system, the 'character set' choice for a 'String Match' now includes the quick-select options for hexadecimal and base64 chars.

When hitting share->copy hashes on a thumbnail, the program now loads up the md5, sha1, and sha512 hashes into the menu labels, so you can now verify md5s or whatever at a glance.

I wrote a thing for API devs and any other advanced users interested in hydrus content updates about how the Current Deleted Pending Petitioned model works in hydrus: https://hydrusnetwork.github.io/hydrus/developer_api.html#CDPP . I went into very specific detail because when talking about this with a user the other day, I couldn't remember it perfectly myself.

Lastly, I am happy to say that I succeeded in doing, as I planned, some meaty background work. The 'Metadata Conditional' object, which does 'does this file have x property, yes/no?', and which I have been thinking about for a couple years, got its first version, and it went a lot better and simpler than I expected. The whole thing plugs into the existing system predicate system and will share much edit UI with it. The MC will be a key lego brick in the development of the duplicate auto-resolution system, and most of the comparison logic for the first version is essentially done now. I pretty much just have to write the database tables and maintenance daemon, and then some UI, so fingers crossed it won't be too long before I am rolling it out for advanced users to play with.

next week

I want to add local media file path fetching to the API and do a bit of API permissions cleanup.
Open

R: 2 / I: 0 / P: 1

Version 589

https://youtube.com/watch?v=Ka4pfP2z8iA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v589/Hydrus.Network.589.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v589/Hydrus.Network.589.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v589/Hydrus.Network.589.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v589/Hydrus.Network.589.-.Linux.-.Executable.tar.zst

I had a good couple of weeks mostly cleaning code and optimising things.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

If you have a lot of files, your database update may take a couple minutes this week. Some users have had unoptimised similar-files search for a while, and this fixes it.

If you have a lot of tag siblings and parents (e.g. if you sync with the PTR), I have reduced the lag around sibling/parent processing significantly. There's a cache that previously had to be regenerated on every sibling/parent change, and for the PTR that could take 1.3 seconds, now it just updates in a couple milliseconds. If I have been working with you on lag issues, and we discovered that shutting off 'normal time' sibling/parent work fixed it, try turning it back on this week and let me know how it goes. If you do prefer manual control of this, there's also a new 'sync now' menu entry under tags->sibling/parent sync that basically slams all the 'work hard' buttons and functions as a catch-all 'do all outstanding work now'.

A bunch of big lists like the 'file log' should now initialise and sort a little bit faster, particularly when you are pushing 10,000+ items.

I fixed the 'page chooser' dialog's 3x3 layout, which was accidentally un-done in a recent layout overhaul. This thing is stupid, but I'm still fond of it. There's also some new checkboxes under options>pages that govern whether 'my files' and the more advanced 'local files' will show up in the 'files' choice.

The safebooru parser should stop grabbing '?' tags, and catbox collection URLs are now parseable.

next week

I want to get some meaty work done, either chipping away at dynamic file storage or duplicate auto-resolution.
Open

R: 5 / I: 0 / P: 2

Version 588

https://youtube.com/watch?v=X7OpjB_8sHQ [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v588/Hydrus.Network.588.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v588/Hydrus.Network.588.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v588/Hydrus.Network.588.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v588/Hydrus.Network.588.-.Linux.-.Executable.tar.zst

I had an ok week. Multi-column lists work faster across the program.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I finished my list rewrite. Multi-column lists look and work exactly as they did before, but they initialise and sort faster. I still have some optimisation to do, but my test list of 170,000 items now sorts in about four seconds. More generally, many normal delete and insert events should have just a little less lag. I hope this makes dealing with large file logs and so on a bit less of a hassle!

Otherwise, I fixed some visual bugs and cleaned up some similar files maintenance code.

next week

I want to optimise some db maintenance code and otherwise just do some simple cleanup.
Open

R: 1 / I: 0 / P: 2

Version 587

https://youtube.com/watch?v=czrmBIHANV4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v587/Hydrus.Network.587.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v587/Hydrus.Network.587.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v587/Hydrus.Network.587.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v587/Hydrus.Network.587.-.Linux.-.Executable.tar.zst

I had an ok week. I didn't have time to finish my big list rewrite, so I'm just rolling out some little jobs today.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I made another stupid typo last week, breaking the tags->manage tag display and search dialog! Fixed now, sorry for the trouble.

The top-right media viewer hover now shows all the local file services a file is in. For most users this will just be 'my files', but if you use multiple file services, I hope this will be a bit cleaner than the spammy labels I removed from the top hover the other week. I think I'm ultimately going to make these into buttons or add checkboxes or something so we can have one-click local service migrations in future.

I cleaned up the code that handles the two resizable splitters/sashes that separate the normal media page sidebar from the preview canvas and the main thumb grid. There was some ugly stuff in there, and I think I have fixed some odd layout problems certain window managers had. That said, this stuff can be temperamental, so if you are on a weird OS and your pages suddenly lay out crazy, please roll back to your v586 backup and let me know.

next week

I will finish this list rewrite. I have all the code pretty much done, and I feel good about it, but I need to do a ton of testing and polish. It should let us view huge lists with far less UI lag.
Open

R: 1 / I: 0 / P: 2

Version 586

https://youtube.com/watch?v=CVAaLlOUD00 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v586a/Hydrus.Network.586a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v586a/Hydrus.Network.586a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v586a/Hydrus.Network.586a.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v586a/Hydrus.Network.586a.-.Linux.-.Executable.tar.zst

Hey, I did a hotfix to fix a stupid bug when moving from videos to images. If you got the release within twenty minutes of this post going live, get the updated v586a above!

I had a great week getting siblings and parents lookups running faster and finishing some long-planned Client API work. The update may take a minute this week!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I am happy with the new siblings and parents dialogs, but unfortunately the fetch jobs were frequently running super slow on the PTR. This has been a long time problem in other sibling/parent places, and I could never really figure it out. This exposed the problem better and I simply put a bunch of time into the database sibling/parent storage structure and search code this week, and I think/hope I have fixed the worst of it. I also fixed the crazy-long lag spikes we were seeing, which was, unfortunately, just me being stupid last week. If you sync with the PTR (or not!) and have had slow sibling/parent lookups (including in places like the tag autocomplete results list), let me know how it goes!

If you have the media scanbar set to hide completely when the mouse is not over it, I think I fixed the issue where it would come up blank if the media was paused while the scanbar was hidden!

The options widgets that are an editable number with a checkbox beside saying 'no limit' are now initialised with a nicer default number when they start with 'no limit' checked. Previously, this stuff was all initialising to 1 every time, which wasn't always helpful if you actually wanted to go edit it. I'm pretty sure all the 'noneable' integer widgets in the options dialog now soft-initialise to the actual defaults those numbers are on a fresh client.

If you use multiple local file services, then when you middle-click a tag in the media viewer, the new search page now correctly retains the original file domain of the media viewer. Although I use multiple local file services myself on my IRL client, I do not browse around in them all that much, so let me know where else this sort of stuff defaults to 'all local files' or 'all my files'.

I have removed a hard limit that said 'don't run an import folder for more than an hour'. If you have a mega import folder with hundreds of thousands of files, let's see how it goes.

If you have done Client API or tag migration work and ended up with some bizarre tags that are both pending (+1) and petitioned (-1) to the same service, check the changelog this week!

client api

Thanks to a user, the Client API call that renders images can now output in jpeg and webp, can change the quality of the output, and will render to a target resolution!

I also think I finally finished off the first full version of 'multiple local file services' Client API support. You can now set a custom import destination for the 'add file' and 'add URL' commands, and you can now copy files from one local file service to another using a new 'migrate_files' call.

next week

I've been working on several things recently that can populate a multi-column list with a hundred thousand or more rows, and it has reminded me that my core list code relies on an old hack in it that makes initialising and sorting such big lists super laggy. I have researched how to improve it and hope to do so next week!

Unfortunately, I do feel myself going down with something, so it might be delayed.
Open

R: 1 / I: 0 / P: 2

Version 585

https://youtube.com/watch?v=LREOmHLII70 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v585/Hydrus.Network.585.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v585/Hydrus.Network.585.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v585/Hydrus.Network.585.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v585/Hydrus.Network.585.-.Linux.-.Executable.tar.zst

I had a great couple of weeks getting the tag siblings and parents dialogs to load quickly.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

fast siblings and parents

The PTR has been a painful success. It is great, and I am grateful for how it keeps growing, but every time we add ten thousand or a hundred thousand new things somewhere, it lags out some bit of UI where I never thought it would be a problem. Anyone who has tried to work with PTR siblings or parents knows what I am talking about--it can take five or ten seconds, every single time, to load the manage tag siblings/parents dialogs. The same is true for anyone who has programmatically imported siblings from a booru--adding 100,000 pairs can be neat, but editing them manually is then a constant frustration.

So! I have rewritten both dialogs, and the long and short of it is they now load instantly. If they need to review some pairs to do logic (like 'hey, does this new pair the user wants to add conflict with the existing pair structure?'), it all happens in the background, usually so quickly you never notice it. We'll see how this actually works on IRL situations, but I do feel good about all this work. There was a lot to do, but it ultimately seemed to go well, enough that I had time for some bells and whistles.

Beyond some optimisations and loop-detection fixes, there's a workflow change in that these two dialogs now have a 'stickier' workspace. The list of pairs has typically shown anything related to what you have waiting to be added, and I have now extended that to say 'and they now stay in view after you add'. Whenever you type in a tag, everything related to that tag is loaded up and stays in view while you work on other things. If you want to clear the workspace, you can just click a button to reset. I hope this makes it easier to edit and even merge larger sibling groups.

This is all a big change, and I'm sure I've messed up somewhere. If you do siblings or parents a lot, give it all a go and let me know how it works out. The PTR really is huge, and some larger groups may still take a second or two to load--we'll see.

other highlights

Hitting escape now deselects any taglist!

options->media viewer gets a new 'Do not allow mouse media drag-panning when the media has duration' checkbox. If you often misclick when scrubbing, try it out.

I pared down the spammy 'added to x service 3 days ago' lines in the media viewer's top hover. It now pretty much just says 'imported, modified'. If you need to see archived time or something, note that the timestamps are still available on the normal media right-click menu, on the flyout submenu off the top row.

next week

I have lots of small things to be getting on with, so I'll just catch up on my normal queue.
Open

R: 11 / I: 2 / P: 2

Version 584

https://youtube.com/watch?v=L5dcODmclFU [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v584/Hydrus.Network.584.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v584/Hydrus.Network.584.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v584/Hydrus.Network.584.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v584/Hydrus.Network.584.-.Linux.-.Executable.tar.zst

I had an ok week working on some small jobs and new Client API commands.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I fixed a bug that was allowing wasteful file re-downloads from Pixiv and Twitter. I accidentally left a hole in the recent changes to the URL 'neighbour-testing' logic, where it tries to determine if a 'already in db/previously deleted' URL determination is trustworthy, and sites where posts can have multiple files were not able to return 'already in db' or 'previously deleted' until the file itself was redownloaded. I have filled the hole in--thank you for the reports, sorry for the trouble, and let me know if you notice anything else weird going on.

This is a weird thing, but in the same way that if you double-click a tag in the normal search page, it adds that tag to the search, if you ctrl+double-click a tag, you enter '-tag'. It was difficult to pull this off due to ctrl+single-click doing a deselect (usually it was best done with ctrl+enter on keyboard), but I've smoothed out the click selection logic and it does, a bit more, what you think it should. Give it a go!

If you ever use 'regex' 'system:url' predicates, they may run significantly faster this week if you mix them with other search predicates. regex URL preds now run absolutely last in the master file search, so they benefit if tags or other system predicates have already reduced the search space. Regex URL search is pretty much the worst-performing part of all hydrus file search, and there isn't much I can do about it, but I did a little research this week and I have a couple of ideas for the future--it might even allow for sometimes-fast regex tag search, but we'll see.

client api

Everything in the hydrus 'pending' menu (e.g. for sending mappings to the PTR) is now available on the Client API, with a new 'Commit Pending' permission needed to do it.

A new command in the file relationships set also lets you remove all the potential pairs for files.

next week

I will make the tag siblings/parents dialogs load instantly. This job keeps on getting put off, but I am determined to finally clear it, even if it takes a couple weeks.
Open

R: 1 / I: 0 / P: 2

Version 583

https://youtube.com/watch?v=-KWxaOmVNBE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v583/Hydrus.Network.583.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v583/Hydrus.Network.583.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v583/Hydrus.Network.583.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v583/Hydrus.Network.583.-.Linux.-.Executable.tar.zst

I had an ok week mostly cleaning code.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

The command palette (ctrl+p by default) gets some new settings under the new options->command palette panel. You can now search for page of pages, and the menubar/media actions, which were previously hidden behind advanced mode, can now be turned on or off separately. I haven't touched this thing much since it was first submitted, but let me know what you would like it to do in future. I know it currently needs some filtering to get rid of unhelpful menu actions, so let me know what is most annoying and I'll see what I can do.

The 'manage times' dialog now lets you set timestamps to multiple domains at once, either the website domains or file services. You can just select multiple rows and hit edit, and you'll set the same time for everything you selected. You can even do a 'cascade'.

If you have weirdly rotated PNGs, try hitting manage->maintenance->regen file metadata on them. I was applying EXIF rotation incorrectly to PNGs. Also, it looks like the 'determine if the file has EXIF metadata' job may have been firing false negatives in the recent past, so if you have been using this in the past few weeks, you might want to run those jobs again.

next week

I want to do some Client API stuff, and probably just some small fixes and quality of life otherwise.
Open

R: 1 / I: 0 / P: 2

Version 582

https://youtube.com/watch?v=eN3UHq6hjBc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v582/Hydrus.Network.582.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v582/Hydrus.Network.582.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v582/Hydrus.Network.582.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v582/Hydrus.Network.582.-.Linux.-.Executable.tar.zst

Hey, if you use the macOS App and use the 'hydrus default darkmode' stylesheet, skip the release this week! I removed it by accident.

I had a great week mostly working on UI fixes and quality of life.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

sidecar UI

The sidecar UI now has a testing panel with a live preview of what is going to be sourced. If you are doing a manual import or managing an import folder, it shows a sample of the files actually about to be imported and what their sidecars will be loading, and if you are doing a manual export or managing an export folder, it shows a sample of the files in hydrus you will be exporting and what tags/urls/whatever you have selected to be exported. If you set up string processing rules, the test panel also shows these.

I hope this helps you see better that you have the sidecar metadata migration set up how you want, but it is only one step. I want to do more: more and better testing panels, and fewer ridiculously nested dialogs, and also in the general parsing and string processing UI, which suffers from the same problems. Let me know how this works for you!

stylesheet colours

The colours in options->colours are now all settable in a stylesheet! The 'colours' options panel remains, but it is now wrapped in a 'override what is set in the stylesheet?' checkbox, which is default True for existing users and False for new users. Selecting a darkmode stylesheet with the override set to False now gives you some good dark colours, instead of the ugly mix of dark and white we had before. Thanks for bearing with me as we figured this out.

If you are a stylesheet maker, please check the hydrus_default.qss to see how this works. I have pasted the internal hydrus darkmode default colours into all the default stylesheets in hydrus, so feel free to go in there and adjust these to something better for your particular stylesheet, and if you send it in I'll be happy to update the defaults.

I am not totally sure what the future of the 'colours' panel is. I suspect it will be migrated into a more dynamic system that has odd custom rules like 'if a file has >4 star rating, give it a gold border'. We'll see how this goes, and we'll see how Qt 7, which I understand has some better darkmode tech, proves itself.

next week

I'd like to take a week to focus on boring code cleanup.
Open

R: 1 / I: 0 / P: 2

Version 581

https://youtube.com/watch?v=Sy5BFkcqqF8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v581/Hydrus.Network.581.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v581/Hydrus.Network.581.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v581/Hydrus.Network.581.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v581/Hydrus.Network.581.-.Linux.-.Executable.tar.zst

I had a great week mostly working on a new tool for tag repository janitors.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

normal stuff

If you use the 'Dark Blue' stylesheet, check out the new '1.1' version this week, created by the original author.

There's a new 'shimmie' downloader, thanks to a user. It has better tag, file, and hash parsing tech. By default, for our purposes, this means r34h and r34@paheal downloaders work better and more efficiently.

I wrote some new tech this week to scan the database for tag filters more efficiently. This now works in the 'migrate tags' system. If you try to do a 'migrate series: tags' or similar, the setup time for that job should now just be a couple of seconds, whereas previously for something like the PTR it could be several minutes.

janitor stuff

There's a new 'admin' menu on all tag right-clicks.

If you have 'change options' permission, you will see which files are blocked or not blocked and can directly edit the repository's tag filter right from the menu.

If you have 'resolve mappings petition' permission, you will see a new 'purge tags' command. This launches a new window that lets you select certain tags to be completely deleted from the repository. No more searching in 'all known files' and then doing a laggy ctrl+a->f3 thing--you can just search up the tags and then fire off the job and it is all done in the background with a custom 'migrate tags' job that queues up the petitions for you, usually in a few seconds. Take care, and let me know how it works out!

Also, there's a new command under services->administrate services, 'purge tag filter', which lets you do this 'purge tags' command on the current tag filter, so there's now an easy way to retroactively sync a tag repository to whatever you have blocked right now.

All this tech works clientside, no need to update the server. I expect to add some bells and whistles in future like showing the current tag count in the 'purge tags' list and offering some sort of 'replace what you delete with tag y' hard-replace tech, and I think I'll provide a similar version of 'purge tags' for normal users so they can queue up big deletes/petitions (i.e. local or repository) all the easier.

next week

I want to make the sidecar UI nicer to work with.
Open

R: 4 / I: 0 / P: 2

Version 579

https://youtube.com/watch?v=qzTwBQniLSc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v579/Hydrus.Network.579.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v579/Hydrus.Network.579.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v579/Hydrus.Network.579.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v579/Hydrus.Network.579.-.Linux.-.Executable.tar.zst

I had a great week mostly working on UI quality of life.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

There are several places where you can enter regex (clever text search rules) in the program. I have written a nicer text input widget and spammed it everywhere. It has several improvements: it colours green/red depending on whether the current text will compile; its menu button collects better tutorial links; and the one in the String Converter regex replace now shows how to do (unnamed) or <named> group replace.

Many multi-column lists across the program have new scroll-to tech. If you import a new parser now, or edit one and change its name, then the list will scroll to the new item once done. Selection preservation through various updates is also improved. The 'manage URL Classes' dialog test input will also select and scroll to the matching URL Class. There's plenty more to do, but I've ironed out a bunch of ugly behaviour.

I've improved some 'hey I think we already saw this URL before' logic, fixing an awkward false positive result. If you have had incorrect 'already in db' on slightly different versions of a file on a booru, and you traced it to a bad source url, they should work in future.

Import Folders now work much more efficiently. Due to some stupid old code, they were wasting a bunch of CPU, particularly when the import folder was very large. If you have big import folders that are normally laggy to load, let me know how your background latency feels this week--it might be that I fixed some weird jitter/hangs here.

I folded in the library updates from the 'future build' test we ran a few weeks ago. There were no problems with the test, so Linux and Windows users get a slightly newer version of Qt, and Windows users get new sqlite and mpv. Let me know if you have any boot problems!

I also fixed a weird recent problem with the 'requirements.txt' that users who run from source rely on to set up their environment. There's a new version of numpy out that breaks some things, so if you tried to set up a new venv in the past few days and could not boot, please try again with this.

next week

Some more slightly-larger-than-small jobs. I have some weird sibling problems to fix, PTR janitor workflow to figure out, and awful sidecar UI to work on. I'd also like to get back to the duplicate auto-resolution system, but I think I'll be out of time.
Open

R: 1 / I: 0 / P: 2

Version 580

https://youtube.com/watch?v=BvfHlZ8QRaI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v580/Hydrus.Network.580.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v580/Hydrus.Network.580.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v580/Hydrus.Network.580.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v580/Hydrus.Network.580.-.Linux.-.Executable.tar.zst

I had a good week working on a mix of stuff.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I may have fixed a program freeze when minimising to tray via the close button (the settings for this are under options->system tray). If you have had trouble with this before, please, when you are at a convenient point to risk a hang, try it again and see if you have trouble. If you do, what happens if you minimise to system tray from the file menu--still have problems, or is that reliably fine?

I added a new maintenance command to tag right-click, 'regenerate tag display'. This is a catch-all job to fix bad autocomplete counts and sibling/parent display. Previously, the way to fix this was to hit a 'regen the whole cache' job under the database menu, but on large clients this could take ages--we now have a way to debug and simply, fingers crossed, just fix a bad tag, all in, typically, a few seconds. If you have any jank siblings or 'the autocomplete said (23-28), but it gave me 8 files', try it out!

I fixed an issue with temp-file cleanup after importing read-only files. If you do a lot of unusual/misc hard drive imports, you might like to shut your client down, check your temp folder (hit _help->about_ to find it), and delete anything called 'hydrusXXXXXXXX'--it is all useless cruft after the client is shut down. It shouldn't come back!

The new 'eye' icon in the media viewer's top hover window now has the 'apply ICC Profile stuff' option. It updates the image live, as you click it! If you are interested in ICC Profiles--and perhaps comparing this in the duplicate filter--check it out. I also added a shortcut for it, to the 'media viewer' set.

The 'file log' has a new 'search for URLs' menu command, which lets you explore what has the selected URLs mapped. It is basically a copy of what I added to the media right-click 'urls' menu recently, and it'll help figure out weird URL collisions and mis-maps.

The Client API '/add_tags/add_tags' command has a couple of clever new parameters to govern how deleted records are navigated. If you are doing clever migrations or other big mappings operations with the API, check it out.

next week

I want to get some repository janitor workflow stuff done.
Open

R: 4 / I: 0 / P: 3

Version 578

https://youtube.com/watch?v=ieKs9G1YBl4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v578/Hydrus.Network.578.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v578/Hydrus.Network.578.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v578/Hydrus.Network.578.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v578/Hydrus.Network.578.-.Linux.-.Executable.tar.zst

I had a good week mostly doing some simple work. In a bonus, animated webps are now fully supported.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Animated webp decoding is not widely supported, but we discovered a method this week and I plugged it into my old (slightly janky) native animation viewer. I have made it work just like the for gif and (a)png, where, within hydrus, images and animations will count as different filetypes. On update, all your existing webps will be queued for a rescan. If they are actually animated, they will become 'animated webp' and get num_frames and a duration, and they'll play animated in the media viewer. Let me know if you run into any trouble with it!

I added an 'eye' icon menu button to the top hover window in the media viewer. It has those five new checkboxes for 'draw stuff in the background' I added last week, and I expect to add 'always on top' and similar options to it in future.

The 'known urls' media menu has a couple changes. It is now just called 'urls'; it gets a 'manage' command, moved from the 'manage' menu; and you can now open an URL 'in a new page' (i.e. opening a new search page with 'system:known url=blah'), so if you need to find all the files that have a particular URL, this is now just one click.

next week

I am now on vacation for a week. As normal I'm just going to disappear to shitpost fake-E3 and otherwise try diligently to achieve nothing of any importance. I'll be back Saturday 15th, with v579 on Wednesday 19th. Thanks everyone, see you later!
Open

R: 3 / I: 0 / P: 3

Version 577

https://youtube.com/watch?v=9QY0OJ8RHYE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v577/Hydrus.Network.577.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v577/Hydrus.Network.577.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v577/Hydrus.Network.577.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v577/Hydrus.Network.577.-.Linux.-.Executable.tar.zst

I had an ok week. There's a mix of small improvements, and some neat OS integration.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Thanks to a user, we have some cool new OS integration this week. For Windows, you can now go open->in another program on a thumbnail to get the Windows open with dialog, and open->properties to get the normal Windows file properties window. Also, the various places that can view a file in your OS file explorer now work better, and are better about selecting the file(s). Users who runs from source will want to rebuild their venvs this week to get this new stuff.

When you have a single 'edit time' panel, like the sub-panels for modified and archived time in the 'manage times' dialog, now accept any date string for the paste button. So, you can now paste something from your web browser or just 'yesterday' or something, and it should all work. This uses the same date parsing tech we have had success with elsewhere.

I broke the 'media' help page into two and added some new checkboxes to set whether to duplicate the text of the different hover windows in the background of the media viewer. If you want a clean background, it is now easy, under the new 'media viewer' page.

I fixed some more URL encoding stuff, this time related to brackets in parameter names, and buffed the 'clear orphan tables' database routine. If I have been working with you on either of these, please try your thing again and let me know how it goes.

next week

This is the last work week before my vacation, so I'll just do some simple cleanup work again.
Open

R: 1 / I: 0 / P: 3

Version 576

https://youtube.com/watch?v=gWmECLnMKGk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v576/Hydrus.Network.576.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v576/Hydrus.Network.576.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v576/Hydrus.Network.576.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v576/Hydrus.Network.576.-.Linux.-.Executable.tar.zst

I had a great week. The program should be less laggy when busy.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

improved file access latency

I reworked the client file manager's locks to be more fine and sophisticated. When you have several importers working in the background, and particularly importers handling large files, the client will delay access to your files and thumbnails far less, and generally not at all. I have been thinking about the specific change I intended to make here for a while, but this stuff can be tricky and I wanted to think and be careful how I did it. All tests so far are proving good, but let me know if you run into any trouble.

Secondly, when you do have file access lag, the mpv player now waits more politely. It won't hang the UI any more; it'll just sit on a black screen until the video file is ready. You can roll off to another media and come back, close the media viewer entirely, whatever you want.

misc

Managing 'import options' in gallery downloaders and thread watchers and their sub-downloaders should be a touch easier this week. I have replaced the static 'set options to downloaders' buttons with a dynamic button that appears only when the current selection of downloaders actually differ with what the main page has set. That pages and their downloaders have unsynced options has always been a complicated concept that I have had difficulty highlighting and explaining, so I hope this improves things a little just by osmosis. I also fixed a bunch of holes and bugs in the system.

Thanks to a user, we now have support for .doc, .ppt, and .xls Microsoft Office formats. A whole bunch of old stuff should now work, and it should get word counts in several cases. This requires a new library, so users who run from source will want to rebuild their venvs to get the tech.

If you noticed that colour picker controls would take several seconds to open and close, it shouldn't happen any more! It was a weird hack I injected to get around a Qt bug, and it seems to be fixed in the current version we use. However, if you find you suddenly cannot select any colours in the gradient square of the colour picker, that it feels like your mouse is drag-clicking the whole time, then I guess the bug wasn't fixed for you, so let me know!

future build

I am making another future build this week. This is a special build with libraries that I would like advanced users to test out so I know they are safe to fold into the normal release. More info in the post here:

https://github.com/hydrusnetwork/hydrus/releases/tag/v576-future-1

next week

I have two more weeks of work before my Summer vacation. I'll try not to start anything too big and just do cleanup and little fixes.
Open

R: 1 / I: 0 / P: 3

Version 575

https://youtube.com/watch?v=Auc5wHXPQaw [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v575/Hydrus.Network.575.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v575/Hydrus.Network.575.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v575/Hydrus.Network.575.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v575/Hydrus.Network.575.-.Linux.-.Executable.tar.zst

I had a great week. I made a bunch of small improvements, and I am gearing up to start duplicate auto-resolution.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

The new tag autocomplete 'children' tab now sorts by count and clips to the top n (default 40) results. You can change the n under options->tags. This takes a bit of extra CPU to figure out, so let me know what performance you see in IRL situations.

When you ask subscriptions to 'check now', if there are a mix of DEAD and ALIVE subs, it now asks you which ones you want to check. Should be easier to resuscitate DEAD subs en masse now.

More twitter URLs are supported by default. You should now be able to drag and drop a vxtwitter or fxtwitter embed URL on the client and it'll figure it all out.

I fixed the (non) alphabetisation of GET URL parameters in URL Class settings, which I broke by accident a few weeks ago with the new URL encoding tech. If you have a super advanced downloader that relies on parameters being in a certain order, please try again and let me know if you still have any issues.

duplicate auto-resolution

This week I also put time into planning out the duplicate auto-resolution system, which will optionally and highly-user-configurably resolve the easiest duplicate pairs without human input. I am happy with my plan, and I have sketched out a skeleton of code. The whole problem looks simpler than I expected, which is good, but as always there will be a lot to do.

I will keep chipping away at this in the coming weeks and will roll out the features in waves for advanced users to try out. The first version will resolve jpeg/png pixel duplicate pairs.

next week

A bit more auto-duplicate resolution work, and I'd like to see if I can reduce some of the juddery latency when you try to look at media, particularly videos, while the client is busy doing imports in the background.
Open

R: 2 / I: 0 / P: 3

Version 574

https://youtube.com/watch?v=XBETVhHpcPk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v574/Hydrus.Network.574.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v574/Hydrus.Network.574.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v574/Hydrus.Network.574.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v574/Hydrus.Network.574.-.Linux.-.Executable.tar.zst

Note: If you are updating from v570 or earlier and you use the Windows or Linux .zip or .tar.zst 'Extract' releases, you have to do a clean install one time to get v571 or later! (https://github.com/hydrusnetwork/hydrus/releases/tag/v571, https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs). If you are a Windows installer/macOS App/source user, you do not need to do a clean install to get over the v570->v571 bump; just update as normal.

I had a good week. The database has some new repair tech, and the new default twitter downloader now gets full-size images. The update this week may take a minute due to some database maintenance.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

For some time, some users who sync with the PTR have had sporadic '404' errors that interrupt downloading/processing. We never pinned down exactly what was causing or fixing this (a bunch of different maintenance routines seemed to fix it eventually) until this past week. Long story short is there was a SQLite issue that was crossing a wire on some program crashes, and a bunch of new repair code now detects and fixes this situation. Several rare 'this file appears to be this file' issues are fixed as a result. Your 'local hashes cache' will do some work on update, but the new code should run pretty fast, so even if you have millions of files, it shouldn't take too long to finish. If you have had PTR syncing problems, please try unpausing after updating this week--while it may fail one more time, the new repair code should be able to fix it now. Let me know how you get on!

I screwed up the new default twitter downloader last week--it was not getting the full-size versions of images! This is fixed today, so if you spammed a bunch of tweet URLs last week, please do this to fix it: do a search for 'imported within the last 7 days/has a twitter tweet url/height=1200px' and then copy/paste the results' tweet URLs into a new urls downloader page. The full-size images should download. Once done, assuming the file count is the same on both pages, go back to your first search page and delete the 1200px tall files. Then repeat for 'system:width=1200px'! Sorry for the trouble!

If some of your files get an odd colour tint, is it always on files under 'system:file properties-&gt;has icc profile'? If so, try unchecking the new 'apply image ICC Profile colours adjustments' checkbox under options-&gt;media. Let me know what you see.

next week

I blinked and it is May already. I think I have four more releases before my vacation week, and it feels like I haven't done anything, so I'd like to get a larger job done. I want to try the first framework of an automatic duplicate pair resolver.
Open

R: 1 / I: 0 / P: 3

Version 573

https://youtube.com/watch?v=1glH_p16WB0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v573/Hydrus.Network.573.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v573/Hydrus.Network.573.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v573/Hydrus.Network.573.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v573/Hydrus.Network.573.-.Linux.-.Executable.tar.zst

I had a good week working on some quality of life and a tagging experiment.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

my experiment

I've written a new tab, 'children', for the tag autocomplete dropdowns. It shows the tag children of anything in the list the autocomplete is managing. The idea here is you have 'evangelion' in your manage tags dialog but you have forgotten the names of the characters--there is now an easy-select list for you.

Since this is a first draft, it is workflow-ugly. It works well when there are only one or two parent tags in the list, but any more than that (or something with hundreds of children like pokemon or azur lane) and it becomes spam. Please have a play with it, especially if you are an advanced user, and let me know what you think. I think I'll filter and sort it according to the top n most-counted tags, and there's probably some simple selection-preservation stuff to add, to better let you run through its list quick-hitting enter over and over (atm it resets the selection position on every change).

other stuff

With the help of a user, I've written a simple twitter tweet downloader. This isn't a full search, just something that can download single tweets so you can drag-and-drop tweet URLs onto the client or paste them into an 'urls downloader'. It can handle video and multi-media posts, and it fetches quote-tweet contents too. I've learned there is a different, more advanced twitter downloader written by another user--if you use this, my new tweet downloader will not be added today, since you have that ability already. I'll see if I can add some of the other downloader's tech to the defaults in the near future.

The archive/delete and duplicate filters now yes/no confirm with you when you select to 'forget' your work.

All the builds should now be compatible with very new (>=0.38.x) versions of mpv/libmpv. If you run from source, you might like to rebuild your venv this week.

I continued to deconstruct the local booru (an old way of sharing files from the client) this week. It now no longer starts its server. If you used this, please note that this Client API project replaces it very well: https://github.com/floogulinc/hyshare

next week

I am pretty exhausted, I'm sorry to say, so I'll keep it simple and just do some more quality of life and little bug fixing work.
Open

R: 1 / I: 0 / P: 3

Version 572

https://youtube.com/watch?v=ZMbBsR6Hc6o [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v572/Hydrus.Network.572.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v572/Hydrus.Network.572.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v572/Hydrus.Network.572.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v572/Hydrus.Network.572.-.Linux.-.Executable.tar.zst

I had a good simple week mostly doing just one thing: overhauling the 'share' menu.

Note: If you are updating from v570 or earlier and you use the Windows or Linux .zip or .tar.zst 'Extract' releases, you have to do a clean install one time to get v571 or later! (https://github.com/hydrusnetwork/hydrus/releases/tag/v571, https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs). If you are a Windows installer/macOS App/source user, you do not need to do a clean install to get over the v570->v571 bump; just update as normal.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

share menu

Like the 'open' menu a couple of weeks ago, the 'share' submenu you see off of the thumbnail or media viewer right-click menu has been given a full pass. The layout is clearer, and exactly the same in all locations, and everything it does is now mappable in the shortcuts system.

Also, you can now copy files' thumbnails as bitmaps from this menu. Should let you do saucenao style lookups on unusual filetypes that have a thumb but no normal image.

The shortcuts here are a bit smarter, too--like with the 'open similar files in a new page', I collapsed the four 'copy xxx hashes' commands down to one command with a dropdown (which has options for blurhash/pixel_hash, if you need them); and the three 'copy xxx bitmap' commands are now one with a dropdown that also has the new 'copy thumbnail bitmap' option. Most of the shortcuts also have a choice between 'do the focused file' vs 'do all selected files', if that is important (existing shortcuts will default to the latter, which was previous behaviour for stuff like 'copy sha256 hashes').

Note: I have removed the 'share on the local booru' command. The Local Booru (an optional way to share files with friends using hydrus) was a fun experiment, but I have decided to finally retire it. I never found enough time to dev it properly (I don't think I've touched it in at least five years), and there are many better ways to share files online, be that a third-party host you simply drag and drop to or a clever Client API solution. So, it is gone from the menu today, and I'll slowly dismantle it over the next few weeks until it is completely removed.

couple other things

There's a new checkbox under options->files and trash that adjusts the 'remove processed files after an archive/delete filter' to also remove the stuff you skipped.

Also, thanks to a user, there's a new 'e621' themed stylesheet under options->style. Users who run from source who want to try this probably have an extra step to make sure this works--check the changelog/options panel help text.

next week

I've been thinking of adding a new tab to the autocomplete dropdown that will show children of the current context (e.g. you are looking at something with 'evangelion', in that tab you'll see 'asuka', 'rei', etc.. for quick selection). I think I will play with that idea and do some more misc small jobs.
Open

R: 2 / I: 0 / P: 3

Version 571

https://youtube.com/watch?v=2umMp1VHWiA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v571/Hydrus.Network.571.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v571/Hydrus.Network.571.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v571/Hydrus.Network.571.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v571/Hydrus.Network.571.-.Linux.-.Executable.tar.zst

I had a simple week working on some quality of life and background stuff. There are special install instructions this week!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

new build

tl;dr: If you use the Windows or Linux .zip or .tar.zst 'Extract' releases, you have to do a clean install! (https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs). If you are a Windows installer/macOS App/source user, you do not need to do a clean install; just update as normal.

The future build test went well, so I am folding the updates into the main build. The above releases are updated from Python 3.10 to 3.11, and Qt (UI) and OpenCV (image processing) are moved to newer versions. There aren't any super important changes, but a bunch of little things should work better or be a bit faster.

Unfortunately, the new libraries cause a dll conflict with v570 and earlier (basically the executable sees the py310 dlls beside the py311 ones and gets confused), so we need to clear the install directory of the old files. Just do the clean install and you should be fine!

You don't have to do it for the other modes because: the Windows installer basically does a clean install every time; the macOS App is always enclosed in its own thing doesn't have to worry about old files; and running from source doesn't care about dlls in the same way, although you might like to rebuild your venv today, just to catch up your own library versions.

If you do have trouble booting v571, then please revert to your v570 backup and let me know what happened! There were no problems in the test that people tried a couple weeks ago, so I'm not expecting anything much, but I'll jump on any reports. Also, if you have been struggling with some annoying menu or drag and drop bug, let me know if the new version of Qt fixed you.
Open

R: 1 / I: 0 / P: 3

Version 570

https://youtube.com/watch?v=Poii8JAbtng [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v570/Hydrus.Network.570.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v570/Hydrus.Network.570.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v570/Hydrus.Network.570.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v570/Hydrus.Network.570.-.Linux.-.Executable.tar.zst

I had a good week clearing simple jobs. There's nothing too clever, just a bunch of little fixes and improvements all over.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

All the tooltips in the program now wrap into a neater multi-line block. No more super long single-line tooltips!

The 'open' submenu when you right-click on the thumbnails or the media viewer has been cleaned up and plugged fully into the shortcuts system. It now looks and works exactly the same in either the thumbnail or media viewer view, and 'open file in web browser', 'open files in a new duplicates filter page', are added to the 'media' shortcut set. What was previously four 'show similar files: 0 (exact)' shortcut commands is now merged (and will be automatically updated in v570) into a single 'show similar files' command that lets you set any distance.

I fixed a stupid thing in manage tag siblings/parents that was too-frequently making it so when you spammed a bunch of pairs at it in one go, it wasn't allowing deletes (it was activating the 'only add' functionality). Should be fixed now, and it'll ask to petition pairs amidst adding others, but let me know if this changes your regular workflow annoyingly. These dialogs are way overdue for a complete overhaul.

I fixed some more URL encoding issues with some help from users. If you have a clever downloader that broke in the past couple of versions, sorry! Try it again, and let me know how it goes.

next week

More of this. I'll clean up the 'share' submenu like I just did the 'open' one, too.

Last week's test 'future build' seems to have gone well, no significant issues reported, so I will integrate that into the main build. Windows and Linux users who use the extract builds will have special 'clean install' instructions for v571, but I'll explain it all super easy when we get there.
Open

R: 1 / I: 0 / P: 3

Version 569

https://youtube.com/watch?v=rAbVhoeWcAs [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v569/Hydrus.Network.569.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v569/Hydrus.Network.569.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v569/Hydrus.Network.569.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v569/Hydrus.Network.569.-.Linux.-.Executable.tar.zst

I had a good week mostly fixing small things.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I fixed some issues with last week's URL improvements. It is mostly advanced stuff, so if you are a downloader maker, please check the changelog.

For regular users who download from certain sites that can produce multiple files from one post (I am told that Pixiv was not affected, but Inkbunny was), I am afraid I made a stupid mistake that meant only the first file of those posts was being downloaded. This is now fixed, so if this affected you and you have a bunch of subscriptions, let's fix it: once you are in v569, load your subscriptions up and sort their queries by 'last check time'. Select what has run in the last week, copy the query texts, and then paste them into a new gallery search page with the file limit set to something like 50. Your client will go over what was missed and fill in any gaps.

I harmonised some search logic related to 'system:duration = 0', '<1', '!=0', and other variants around the zero edge case, for duration and the other simple predicates I updated a while ago. When I first wrote this, I tried to thread a needle of logic where in some cases it would return files with no duration and sometimes it would not, but it all ended up a mess, so, from now on, searching for any simple quantity that would include '0' will now include any file that has no value for that property. (e.g. an mp3 has no width, so now system:width < 200 will include mp3s).

If you are a Linux user who uses a hydrus.desktop file, check the changelog! You'll probably want to rename it, and it might even fix your program icon.

I fixed the 'remember last used default tag service in manage tag dialogs' checkbox and dropdown in options->search. It wasn't saving right, sorry for the trouble!

future build

I have a new 'Future Build' ready. This is a build the same as the one above, but it uses newer python and libraries. We do not know if those libraries cause any dll conflicts on any OSes, so if you are an advanced user and would like to help me test it, please check out this link:

https://github.com/hydrusnetwork/hydrus/releases/tag/v569-future-build-1

next week

More small jobs. If I can, I'd like to focus on some UI quality of life.
Open

R: 5 / I: 0 / P: 4

Version 568

https://youtube.com/watch?v=PgqmqeH0iDs [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v568/Hydrus.Network.568.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v568/Hydrus.Network.568.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v568/Hydrus.Network.568.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v568/Hydrus.Network.568.-.Linux.-.Executable.tar.zst

I had a good couple of weeks mostly figuring out better behind-the-scenes URL handling.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

urls

For normal users, tl;dr: URLs better now, you don't have to do anything.

I made a dumb mistake when I first created the downloader engine in how I handle URLs behind the scenes, and today it is fixed. You don't have to do anything, and you shouldn't see any big changes, but in general, URLs and query texts with unusual characters should work better now. You may also notice a URL in the file log or search log will appear one way, e.g. with japanese kanji, but when you copy it to clipboard, it is all encoded to %EF kind of stuff--your browser address bar probably works the same way. It should just mean things copy between your hydrus and other programs with fewer hiccups.

If you regularly use some very odd URLs or downloaders, there might be a hiccup or two. A file that relies of strange encoded characters in its known urls might redownload one time if a subscription hits it, or one extremely strange query text (it'd be a single tag query that has a % character somewhere in the middle) might need to be renamed to %25. If you have a crazy custom downloader that relies on %-encoding tech, please pause it before you update, and then do a test afterwards--it may be the hack you added for the old system is no longer needed. In any case, let me know how you get on, and if we run into a problem with a certain downloader, we'll fix it together. Overall, I'm hoping that working with encoded URLs exclusively will make more complicated downloaders easier to figure out in future, rather than having to fight with odd characters all the time.

I decided to cancel this release last week because I wasn't confident on how I was handling the advanced edge cases of encoding here. I am happy I did, because while all the 'just download from a booru' style downloaders (like the defaults) were fine, the most experimental downloader situations (that e.g. needed encoded parameter keys) needed more work. If you are a downloader maker, then you will be in the guts of these changes, so please check out the changelog. There's new UI in 'manage url classes'--path components and parameters now have their own edit panels with linked text boxes that show the normal and encoded values of the stuff you are entering, and there's also new 'ephemeral token' tech that lets you decide in what ways undefined parameters should be clipped before the URL being sent to the server. The idea of the 'request URL' being different to the 'normalised URL' is broadly elevated and exposed across the program.
Open

R: 3 / I: 0 / P: 4

Version 566

https://youtube.com/watch?v=PlvK2pabBqI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v566/Hydrus.Network.566.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v566/Hydrus.Network.566.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v566/Hydrus.Network.566.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v566/Hydrus.Network.566.-.Linux.-.Executable.tar.zst

I had a good week. The long-awaited incremental tagger is ready, and the program supports some more document types. You will get a yes/no on update, but it isn't a big deal.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

incremental tagger

When you open the manage tags dialog on several thumbnails, it now has a '±' button that lets you tag the files 'page:1', 'page:2', 'page:3' and so on. You can set the namespace (or no namespace), the starting point (so you can start at 'page:18' if you need to), the 'step' (so you can count by +2, or even -1 to decrement), and even prefix/suffix for the number if you need to decorate with 'page:x (wip)' or something.

I'm quite happy with how this worked out. There's some live text that gives you a preview of what's about to happen, so the best way to get to grips with it is to play with it. Just click and poke around, and let me know how you get on. Namespace will be remembered between opens, and if the first file in your selection has a number tag for that namespace, it'll set the 'start' position to that value. If you are setting page tags to a bunch of chapters, or gaps in a larger body, a bit of prep/overlap may help things here.

We've wanted this for years and years, and while I'm not expecting the program to handle paged content beautifully now, this is a decent step forward.

docx and friends

The document types .docx, .xlsx, and .pptx, which are the newer Microsoft Office 'Open' formats, are now recognised by the client. These are secretly zips, so there's a chance you have some in your client already. On update, you'll be asked if you want to scan your client's existing zips to see if they were really one of these. You probably don't, so it isn't a big deal, but if you think you do, hit yes and they should appear.

next week

Cloudflare are rolling out some annoying new cache-optimising tech that is causing file dupes, so I need to work on URL handling so we can patch the most-affected sites.
Open

R: 1 / I: 0 / P: 4

Version 565

https://youtube.com/watch?v=lgpD6OsHCKU [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v565a/Hydrus.Network.565a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v565a/Hydrus.Network.565a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v565a/Hydrus.Network.565a.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v565a/Hydrus.Network.565a.-.Linux.-.Executable.tar.zst

I had a simple week. Lots of small changes today. The update step may take a couple of minutes.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

You are going to get a couple yes/no dialogs on update this week talking about deleting some mis-parsed URLs. If you do not manually store weird data in your 'known urls' store, just click yes. If you have lots of URLs, the work will take a couple of minutes.

options->sort/collect now has four different default tag sort widgets! You can set the default tag sort for search pages, the media viewer, and the manage tags dialogs launched off them.

There's a new 'media' shortcut action 'copy file known urls', that copies all the known urls of your current media selection.

Sidecars set to import to file notes now have an optional 'forced name' field, so if you ave a .txt file with only note text, no name, you can now force it. Some of the UI is less jank here too.

The tag filter UI also got a little polish. There's less logic jank, better labels and tooltips, and you can now copy listed namespace entries to the clipboard and get something you can paste back in elsewhere.

next week

I did not get to the manage tags dialog incremental tagging thing this week, so I'll try again.
Open

R: 1 / I: 0 / P: 4

Version 564

https://youtube.com/watch?v=Oo0o84-TJTU [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v564/Hydrus.Network.564.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v564/Hydrus.Network.564.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v564/Hydrus.Network.564.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v564/Hydrus.Network.564.-.Linux.-.Executable.tar.zst

I had a good week, several system predicates have better range-based searching.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

The system predicates for width, height, num_notes, num_words, num_urls, num_frames, duration, and framerate now support two different kinds of approximate equals (≈): absolute (±x), and percentage (±x%). Previously, the ≈ secretly just did ±15% in all cases, but now you set how and how far they go.

Also, any 'system:framerate' predicate that was '=' will now be converted to ±5%, which is what it was secretly doing before, and any 'system:duration' '=' predicate will also be converted to ±5%, which is what it really should have been doing before. 'system:duration' also allows hours/minutes input, for longer videos.

This predicate overhaul was an important cleanup job, replacing a ton of hacky ancient code with something that is easier to update and maintain. I've collapsed all these preds down to a lot of shared UI and logic, so let me know if there are any display/search quirks, but once we have it nailed down, I hope to replicate this work for the more complicated system predicates.

I reworked what 'Space' does in the media viewer by default. I am updating existing users too, so you'll probably get a little popup about it when you update. Essentially, if you are still on the default shortcuts, Space will now only send 'pause/play media'. It no longer does 'yes' on the archive/delete filter or 'this is better' on the duplicate filter. If you want to go back to how it was, sorry for the trouble--hit up file->shortcuts to set it back.

Thanks to a user, space also does a new 'Quick Look' for macOS users on thumbnails. Try it out! If you are a 'running from source' macOS user, make sure to rebuild your venv this week, or it won't work!

next week

I'd like to figure out incremental tagging on the manage tags dialog, so you can select 20 files and tag them page:7 through to page:26 in one step. Let's see how it goes.

R: 5 / I: 4 / P: 4

html sidecar attempt

Hey,

Since my last message, I recently had time to work with Hydrus, and I developed a semi-functional parser for my HTML files. However, I don't want to include the description string in the tags, so I need to remove it. However, I'm not sure how to do this. I assume that it has something to do with the "string selector/slicer" component, but I'm not sure how that component works. I managed to do an incredibly crude filter by regex filtering by ^[^<]+$, which blocks the < in html tags, but this will presumably fail for any description that doesn't include html tags, for whatever reason. I would really appreciate your advice on this. If you need to see my (bad) code, I have attached the code to copy as a text file.

Thanks!

Ps: I can't import sidecar parsers through pngs, as the file selection dialogue doesn't show png files, just folders. Is there something I'm doing wrong?
Open

R: 1 / I: 0 / P: 4

Version 563

https://youtube.com/watch?v=XXlzWhBvYwg [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v563/Hydrus.Network.563.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v563/Hydrus.Network.563.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v563/Hydrus.Network.563.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v563/Hydrus.Network.563.-.Linux.-.Executable.tar.zst

I had a good week. There's a mix of small fixes and improvements.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

macOS

A user has improved the macOS release in many ways, mostly in brushing up the App to normal macOS standards. The menubar should now plug into native global bar, with some standardised command labels. The program icon is better, some colours should be improved, dialog menus are no longer a crazy hack by default, and the system tray icon is turned on. In a side thing, I also added Cmd+W to close pretty much any dialog or non-main-gui window in macOS, just like hitting Escape.

I think we can expect more work here, so let us know how you get on!

all misc

In more shortcut news, I've added Alt+Home/End/Left/Right to the 'thumbnails' defaults to perform the new thumbnail rearranging. Existing users will also get these! (assuming it doesn't conflict with something you already have set)

Also, the shortcut system now, by default, converts the 'numpad' variants of non-number key presses (think 'numpad Home' or 'numpad Return') into the non-numpad 'normal' ones. You now only need to map 'Left' or 'Delete' once when setting a shortcut, and, no matter how crazy your keyboard's internal mapping is, it should just work. If you do need to differentiate between the numpad and normal variants of these keys, you can turn this new behaviour off under options->shortcuts.

The menu on a 'file log' button now lets you delete all the remaining 'unknown' items or set them all to skipped.

I fixed another damn problem with copy/pasting timestamps into the manage timestamps dialog. When you paste a timestamp, it should 'stick' better now!

If you have had problems with mpv sometimes going silent on 'every other video' or had your windows being rescued from 'off screen' even though they were supposed to appear just on some monitor, check the changelog for some special new BUGFIX options.

next week

I want to put some work into system predicates, specifically startng with 'system:duration'. The aim is to eventually get rid of all the +/-15% '~=' stuff and replace it with actual customisable values, along with better behind the scenes storage of that data. We'll see how it goes!
Open

R: 1 / I: 0 / P: 4

Version 562

https://youtube.com/watch?v=XnwHtzBf-c8 [Embed]
(go to 4:44:00 for a wild fight)
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v562/Hydrus.Network.562.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v562/Hydrus.Network.562.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v562/Hydrus.Network.562.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v562/Hydrus.Network.562.-.Linux.-.Executable.tar.zst

I had a tepid week, but there's some decent fixes and quality of life improvements.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

all misc this week

I fixed a stupid typo error in the manage times dialog when you go into a single time and try to copy/paste the timestamp. The buttons also add millisecond data now.

Fingers crossed, drag and drops of thumbnails and page tabs will feel snappier this week.

You might see some heavy 'analyze' database maintenance coming down the pipe. Let me know if it proves annoying, or if you even notice it at all--my hope is to iron out some super slow PTR-based tag updates that hit some users, but we'll see how it goes.

If you are an AUR user or otherwise updated to a very new Qt version (6.6.1) and suddenly the column widths of multi-column lists all went ~100px wide, I think I've fixed it! If you were affected by this, I can't recover your old settings, but recall that you can right-click any list header to reset the widths to the default.

next week

I was short on work time this week, so I'll try to hack away at github bug reports again.

R: 2 / I: 0 / P: 4

html sidecars - possible?

Hey,

I was recently trying to import some files into hydrus that I had downloaded using gallery-dl. Thanks to a config-file mistake, all the tag files were saved as html pages. Is there a way to use regex to extract the tags? I attached one of the html files, if that helps.

Thanks!
Open

R: 1 / I: 0 / P: 4

Version 561

https://youtube.com/watch?v=GuVnZZ1sFIc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v561/Hydrus.Network.561.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v561/Hydrus.Network.561.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v561/Hydrus.Network.561.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v561/Hydrus.Network.561.-.Linux.-.Executable.tar.zst

I had a mixed week. Thumbnail rearranging is added, and some bugs and quality of life issues are cleared.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

thumbnail rearranging

So, if you right-click any thumbnail, you'll now see a 'move' menu. This makes it simple to rearrange your thumbnails. You can move your current selection to the start, the end, left one, right one, or 'to here' if you have multiple selected. If your multiple selection is non-contiguous, it will be made so on move, with the move focusing around the position of the first selected item.

You can also map these commands to keyboard shortcuts under the 'thumbnails' shortcuts set. I haven't added any default shortcuts for this yet, but let me know if and what you would prefer--I've been playing around with ctrl+numpad numbers on my dev machine, and it feels nice to me.

In future I'll try and figure out mouse drag-and-drop rearranging. It might have to wait for a larger pending rewrite of the thumbnail grid though--we'll see.

other highlights

Unfortunately, there were a couple of stupid typos in the content processing changes last week. One in the duplicate filter (which the v560a hotfix addressed), and another fixed today that was causing 'already in db' results to not get metadata updates correctly. Sorry for the trouble, and thank you to the users who reported these.

Ctrl+C/Ctrl+Insert is now hardcoded to copy tags from the taglist.

The thumbnail and media viewer menus should now be much thinner. I hate how wide they can get, how annoying it is to hit their many nested submenus when they get like that, so let me know if they still go crazy in some situations.

There was a bitrot issue in v559, the millisecond timestamps conversion, that made it impossible/bad to update from a much older version. This has always been a tricky technical issue to talk about and communicate to the user, so I've now written a better in-client error reporting process that stops the user before the update is even attempted. The upshot this week is that if your client is v551 or older and you try to update to v561 or later, you will be told to update to v558 first.

In lieu of a proper rewrite, I've made some semi-hacky newline processing improvements to the parsing system. If you make downloaders and hate having to deal with extra whitespace in multiline content, notes or otherwise, check out the full changelog and let me know how you get on with it all.

next week

I want to work on github bug reports, which I haven't put proper time into for too long!
Open

R: 3 / I: 0 / P: 4

Version 560

https://youtube.com/watch?v=UBYThAP_C5A [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v560/Hydrus.Network.560.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v560/Hydrus.Network.560.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v560/Hydrus.Network.560.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v560/Hydrus.Network.560.-.Linux.-.Executable.tar.zst

I was suddenly without internet for a while, so there was no release last week, but everything is back to normal now, and I had a great couple weeks' work. The 'edit times' dialog can now handle multiple files.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

editing times for multiple files at once

If you launch 'edit times' on a single file, everything is the same. If you open it on a larger selection, it will now show, in a summarised way, the range of each particular time type, and how many files in the selection have or do not have a time set there. The controls all work as before, but in general, when you set a time, all files are now locked to that new time. Give it a play and you'll see how it all works.

SInce we don't always want to set exactly the same time to a set of files, but rather ideally a similar, staggered time, there is also a new 'cascade step' system in the multiple file version of the dialog. Every time-edit you open up has the ability to enter a little step, say 100ms, which will cause the dialog to set each successive file in the selection to be that much later (or earlier, negative values are allowed!) than the last. This way, if you have a bunch of files of something contiguous, like a comic chapter, that all have different, merged, or otherwise awkward import times, you can now manually sort them in the UI using tags or hackery and apply a new import time based on the first file plus a few milliseconds each time. Thereafter, any time you sort those files by import time, they'll also be nicely in page order time, just as if you had imported them all together from one directory.

I am quite happy with the new dialog, and I plan to copy some of the new techniques I figured out to other dialogs for similar better multi-file handling. I'd also really like, as we have discussed before, to tackle a 'cascading' tagger in 'manage tags', so we can quickly set 'page:n' tags to many files at once.
Open

R: 3 / I: 0 / P: 5

Version 559

https://youtube.com/watch?v=zBuJD-ugT_g [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v559/Hydrus.Network.559.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v559/Hydrus.Network.559.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v559/Hydrus.Network.559.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v559/Hydrus.Network.559.-.Linux.-.Executable.tar.zst

I had a great week working mostly on one thing: converting the timestamps in the program from seconds to milliseconds.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

milliseconds

tl;dr: time numbers better, you don't have to do anything

Since the program started, it has tracked the various file 'times', like import and archive time, using a system that stores time with second precision. It knows that file x was imported at 2023-12-02 14:04:51, but nothing finer. This is common in many computing applications, but sometimes you want more. Particularly, in hydrus, whenever you have a fast importer that brings in two files within the same second, the program does not know, later, which was actually imported first (e.g. when you sort by 'imported time'). This can be annoying when you import a whole chapter of a comic and don't have good page tags set up yet--random pairs of pages will be flipped whenever you next try to sort those files by 'import time'.

So, this week I converted the database to use a time system that has millisecond resolution. Any files imported from now on, or deleted, archived, modified, or 'last viewed', will now get a millisecond timestamp. You don't have to do anything, and nothing outside of the 'edit file timestamps' dialog will appear any different, but it may take a minute to update.

This work was quite simple, but there was a ton of it, years and years of system build-up to go through. Rather than make an already messy system more complicated with a rough injection, I decided to invest the time and clean everything as I moved to the new format. I've tested it back and forth, but given the number of changes, I wouldn't be surprised there is a typo in there somewhere--if you see a file saying it was last viewed '54 years ago' or similar, let me know!

Also, the Client API can now edit timestamps, including with the new millisecond precision. If you are an API dev, check out the new call (and 'edit file times' permission) here: https://hydrusnetwork.github.io/hydrus/developer_api.html#edit_times_set_time

misc

Just a side note, I have removed the sankaku downloader defaults from the program, so new users won't see them. None of them were working well/at all, especially in recent weeks. If you want to grab from complicated sites, and there isn't a clean solution on the shared downloader repo here, https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts , I recommend going with a more robust solution like gallery-dl/hydownloader or just finding the content elsewhere.

If you ever run something like 'system:known url: regex = blah.com/(regex stuff)' and are annoyed at how slow it is, try adding a 'system:known url: domain = blah.com' pred in addition. This combination will now be explicitly much faster than just the bare regex (previously, it could be faster, by accident).

next week

Now we have ms timestamps and nicer code, I'd like to try tackling an 'edit timestamps' dialog that works for multiple files, including a 'cascade' command for force some clever sorts.
Open

R: 5 / I: 0 / P: 5

Version 558

https://youtube.com/watch?v=crJst7Yfzj4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v558/Hydrus.Network.558.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v558/Hydrus.Network.558.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v558/Hydrus.Network.558.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v558/Hydrus.Network.558.-.Linux.-.Executable.tar.zst

I had an ok week. I figured out 'system:number of urls', and you can now import rtf files!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

So, you can now search for 'num URLs'. You will find it under a new stub, 'system:urls', at the bottom of the normal system predicate list (where 'system:known urls' has also moved). This first version is simple--it counts all URLs, regardless of how important, but I can see a future version having the ability to scan by 'post URLs' or specific URL classes. Give it a go and let me know if you run into any trouble!

Also, thanks to a user, you can now import rtf files! There's no word count yet, but I think it should be doable in future.

If you select multiple tags, you can now hide or favourite them all at once. There's a bit of workflow and presentation improvement here, too.

The old 'tag suggestions' box called 'favourites' is now called 'most used'. It was stupid to have two tag systems called 'favourites', so this is now fixed. You still edit them under options->tag suggestions; they just have a more precise and less confusing name now.

If you are an advanced macOS user (i.e. you know how to build something with xcode), you might like to check out Hydra Vista, a new user-made macOS App that presents your client (via the Client API, e.g., perhaps on another computer) in a booru-like wrapper. https://github.com/konkrotte/hydravista

next week

I would like to add timestamp editing to the Client API.
Open

R: 6 / I: 0 / P: 5

Version 557

https://youtube.com/watch?v=xNKXyr6YN8Y [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v557/Hydrus.Network.557.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v557/Hydrus.Network.557.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v557/Hydrus.Network.557.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v557/Hydrus.Network.557.-.Linux.-.Executable.tar.zst

I had a good week back after the holiday. There are some bug fixes and improvements to system:hash parsing.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Tag filters (which operate the various tag whitelists/blacklists across the program) now edit much much faster when they are full up with stuff. You can paste 5,000 items into an empty tag filter in less than a second now, and removing one or a handful of items from a filter with thousands of items already in is now instant. Previously, these things could take many seconds or even minutes due to inefficient overhead.

CBZs that have four or fewer pages should now be recognised correctly.

When you copy 'system:hash' and the 'system:similar to' predicates to clipboard, you now get a much longer string that includes all the hashes in the predicate. These copied strings are all parsable, meaning you can paste them into the same client elsewhere or a different client entirely and it should all just work in a complete loop. The human labels are updated to give you more information, too.

There may be other predicates that are currently parsable if you type carefully but do not themselves copy a parsable string, but I am not sure which they are, so if you run into one, let me know and I'll see what I can do.

If you run from source on Windows--or you'd like to--but you haven't installed the extremely convenient but bulky-to-install 'Git for Windows', I had to go through the installer again to set up my new dev machine, and I wrote out a guide for the full 12-page wizard here: https://hydrusnetwork.github.io/hydrus/running_from_source.html#core It is actually easy to do, and almost everything can be left as default and things will be fine. In any case, Git is great and lets you update in about three seconds, so if you run from source on Windows, give it a go!

next week

My old dev machine died just before Christmas, but I bought a new one and am back to normal work. For next week, I'd like to get some sort of system:num_urls figured out.
Open

R: 3 / I: 0 / P: 5

Version 556

https://youtube.com/watch?v=e3mkeK6pRaY [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v556/Hydrus.Network.556.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v556/Hydrus.Network.556.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v556/Hydrus.Network.556.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v556/Hydrus.Network.556.-.Linux.-.Executable.tar.zst

I had an ok week. I fixed some bugs and added a system to force-set filetypes.

You will be asked on update if you want to regenerate some animation thumbnails. The popup explains the decision, I recommend 'yes'.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

forced filetype

The difference between a zip and an Ugoira and a cbz is not perfectly clear cut. I am happy with the current filetype scanner--and there are a couple more improvements this week--but I'm sure there will always be some fuzziness in the difficult cases. This also applies to some clever other situations, like files that secretly store a zip concatenated after a jpeg. You might want that file to be considered something other than hydrus thinks it technically is.

So, on any file selection, you can now hit right-click->manage->force filetype. You can set any file to be seen as any other file. The changes take affect immediately, are reflected in presentation and system:filetype searches, and the files themselves will be renamed on disk (to aid 'open externally'). The original filetype is remembered, and everything is easily undoable through the same dialog.

Also added is 'system:has forced filetype', under the 'system:file properties' entry, if you'd like to find what you have set one way or the other.

This is experimental, and I don't recommend it for the most casual users, but if you are comfortable, have a play with it. I still need to write better error handling for complete nonsense cases (e.g. calling a webm a krita is probably going to raise an error somewhere), but let me know how you get on.

other highlights

I fixed some dodgy numbers in Mr. Bones (deleted file count) and the file history chart (inbox/archive count). If you have had some whack results here, let me know if things are any better! If they aren't, does changing the search to something more specific than 'all my files'/'system:everything' improve things?

Some new boot errors, mostly related to missing database components, are now handled with nicer dialog prompts, and even interactive console prompts serverside.

I _may_ have fixed/relieved the 'program is hung when restored from minimise to system tray' issue, but I am not confident. If you still have this, let me know how things are now. If you still get a hang, more info on what your client was doing during the minimise would help--I just cannot reproduce this problem reliably.

Thanks to a user who figured out all the build script stuff, the Docker package is now Alpine 3.19. The Docker package should have newer libraries and broader file support.
Open

R: 1 / I: 0 / P: 5

Version 555

https://youtube.com/watch?v=BE4ptZRUJZo [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v555/Hydrus.Network.555.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v555/Hydrus.Network.555.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v555/Hydrus.Network.555.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v555/Hydrus.Network.555.-.Linux.-.Executable.tar.zst

GET

I had a good week. There's some cbz/Ugoira follow-up, nicer system:time parsing, and much better boot error-handling.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

The CBZ/Ugoira stuff last week went ok! We had a few too many false positive Ugoiras, so that test is tightened up, and those files should soon become CBZs. For false positive CBZs (i.e. just a zip of CG images or similar), we don't have a good automatic solution, but I still plan to roll out 'force this file to be this filetype' tech in future so we can, amongst other things, assign these fixes manually.

The various 'system:archived since xxx' predicates' parsing is now plugged into the same excellent date parser we are using in the downloader system. If you type them in, they'll now take all sorts of date phrasing. Try 'since 01/05/2016' or 'before june 2022' into the autocomplete--you may even be able to use your own language!

There is still a little work to do with the 'since/before x time units ago' variants, though. In some unclear cases (including foreign languages), the before/since may be flipped to what you type. Also, I have decided to soon migrate these predicates to just store days and hours (no more year/month). You can still enter '1 year ago' with this new parser, but on my end, trying to calculate leap years and weird month durations has caused too many problems, so I am going to simply pull back over the near future and let you put 365 or 30 in yourself! In any case, give this stuff a go and let me know how you get on.

When the hydrus client fails to boot really early on, before the main UI system is live, it should now nonetheless pop up a dialog saying what happened! The only way this will fail is if the problem with the boot is the Qt UI library, lol. In either case, the 'hydrus_crash.log' file is still made on your desktop.

After talking it out with users, I have decided to move towards dropping the image library OpenCV from the program. It has served us well, but it is often difficult to install and a bloat, and our flexible alternative, Pillow, works extremely well these days. I'm not ready to flick the switch yet, but we have done work here and there, and if you would like to help me test this out, please hit the 'IN TESTING: Load images with PIL' checkbox under options->media and let me know if you have any images that suddenly load incorrectly.

The String Splitter and Joiner objects in the parsing system now accept \n and \t for newline and tab. If you need to split or join by \, use \\. To not break any existing parsers, existing objects that have a \ have been updated to have \\.

next week

I only have two more weeks in the year, so I'll just do some cleanup and little jobs.
Open

R: 1 / I: 0 / P: 5

Version 554

https://youtube.com/watch?v=HUVtoWIeyp8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v554/Hydrus.Network.554.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v554/Hydrus.Network.554.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v554/Hydrus.Network.554.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v554/Hydrus.Network.554.-.Linux.-.Executable.tar.zst

I had an excellent week. Some important bugs are fixed, and we have some basic support for CBZ and Ugoira files.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

bugs

I screwed up two important calls last week, in fixed-period checker timers and job statuses. If you had watchers not start, subscriptions with incorrect check times, and/or popup messages that threw display errors or wouldn't auto-dismiss, I am sorry! I didn't plan the changes here properly, and several things slipped through my tests.

All the affected systems have been given proper rework this week and have some unit tests to make sure this doesn't happen again. Please let me know if you still have any problems.

animations

I fixed up more of the 'while checking for transparency, this file produced x error!' issues. Checking GIFs for transparency should be a bit faster and more fault tolerant too, now.

Also, the native GIF renderer has much improved transparency support. The multilayered 'noclip' artifacts should be gone, and damaged GIFs will recover better. GIFs also now get thumbnails that are x% in with proper transparency.

Also, APNGs now get transparency: when rendering in the native renderer; in their thumbnails (which are also now x% in); and for 'has transparency' checks.

CBZ and Ugoira

I am adding basic recognition and thumbnails for these filetypes today. Behind the scenes, both formats are essentially just zips with a list of images, so all your zips will be scanned, and if they look like a CBZ or Ugoira, their filetype will change and they will get a thumbnail. Ugoira thumbnails will be x% in, like for other video.

Also, a user is working on true Ugoira rendering now, so I hope we will be able to finally roll this out in the medium term.

Unfortunately, neither format has a particularly definitive/unique specification, so while I have tried to be careful, my tests here are imperfect. We can expect a few incorrect determinations one way or the other. If you get an outrageous false positive or false negative here (e.g. something you know is a Ugoira that stays as a ZIP, or a ZIP of misc files that detects as a CBZ), please send in the details, and I'll see if I can tweak my tests.

next week

I put in extra time this week to figure out CBZ, so I'll let things breathe and just catch up on simple, small work. I have a bunch of interesting quality of life UI items in my immediate todo, so I'll probably focus on that!
Open

R: 1 / I: 0 / P: 5

Version 553

https://youtube.com/watch?v=FhGziRJY730 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v553/Hydrus.Network.553.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v553/Hydrus.Network.553.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v553/Hydrus.Network.553.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v553/Hydrus.Network.553.-.Linux.-.Executable.tar.zst

I had a great week. The issues with gifs are fixed, and slideshows with videos are smoother.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I was happy with the 'has transparency' work last week, but unfortunately some damaged animated gifs (usually where one frame was borked) were raising a serious error when then were inspected. This error, which was interpreted as a hard drive fault, was a false positive! If you got it (should have been a popup talking about an 'I/O Error', and that maintenance work would halt until the next boot), do not worry--it wasn't a big deal. I have fixed up the error handling and a bunch of other gif issues. Sorry for the trouble!

There are several new slideshow timing options under options->media. You don't need to tweak them unless you really care, but now, when you do a slideshow that involves video or audio, the transitions should happen earlier or a little later in order to line up with where the video naturally ends. Check the changelog and tooltips of the new widgets for full details.

The various 'import options' on downloaders now say '(all default)', '(some set)', and '(all set)' on their labels, so you can quickly, at a glance, see what you have set where.

The thumbnail manage->regenerate menu is now called maintenance, and it now lists all the possible file maintenance commands. Should make it easier to fix weird problems.

next week

I have some server work to do. Hopefully clear out some github bug reports too.

I also just realised we only have about four more releases before the year is done--it feels to me like the second half of this year has flown by. I'll try and fit in one more medium-sized job before then.
Open

R: 1 / I: 0 / P: 5

https://youtube.com/watch?v=VE2NCPnULA0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v552a/Hydrus.Network.552a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v552a/Hydrus.Network.552a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v552a/Hydrus.Network.552a.-.macOS.-.App.dmg
linux
tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v552a/Hydrus.Network.552a.-.Linux.-.Executable.tar.zst

I had a great week. There's a bunch of small fixes and improvements, and the addition of 'system:has transparency' search.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

has transparency

So, the database can now remember if a file has transparency. This might mean a file with fully transparent pixels, or it might just be an area that is semi-transparent. You can now search it under the new 'system:file properties' entry, selecting 'system:has/no transparency' (or you can just type it).

Like 'has exif' and others before it, 'has transparency' will be correct for all new files instantly, but figuring it out for your existing files will take a bit of background maintenance work. It will take some time for the full results to populate. You can review how much it has still to do under database->file maintenance->manage scheduled jobs.

Note that this is not a 'this file is RGBA' test. There are a bunch of files out there with an alpha channel that is all 100% opaque, so my 'has transparency' check actually looks for impactful transparency information in the image. I'm also keeping it simple to start with, so we are scanning all images except jpegs and animated gifs. No Krita or PSD or anything yet--I'm not sure if our various rendering hacks are even able to pass along accurate alpha info.

As always, I'm interested in seeing any unusual files that fail the test. However, while doing this work, I encountered several files that looked normal but still got the 'has transparency' label. When I inspected them closer, I discovered they had a single 98% opaque pixel, or a border with a slight fade, or just one invisible corner pixel. Maybe some of these pixels are secret artist markers, or perhaps they are tool errors or drawing tablet smudges. They probably aren't really what we are interested in finding with a 'system:has transparency', so be on the watch for them and let me know how bad the problem is in an IRL client. Maybe I can fine-tune this system to say 'image is at least 0.3% transparent'.

other highlights

I fixed a stupid logical typo in the folder move/copy tech changes from last week. If you couldn't run an internal backup or migrate your hydrus folders around, sorry for the trouble! Should all work correct again.

I fixed the 'open externally' thumbnails position if your thumbnail supersampling is >100%, and made it so files without thumbs (e.g. zip, epub) will show their filetype thumb or the hydrus icon.

'system:number of character tags > 4' now parses if you type it in (previously, this didn't support the namespace filter). 'unnamespaced' should work too.

'system:has audio' and the 'system:embedded metadata' stuff, which never had good homes, are now all merged under that new 'system:file properties' entry. If you can't find something weird, check there!

next week

Everyone around me is sick, and I can feel myself going, so it may be up in the air! In any case, I think I'd like to take a simple code cleanup week. Nothing too clever or amazing, but should be some small fixes and QoL.
Open

R: 1 / I: 0 / P: 5

Version 551

https://youtube.com/watch?v=J5I00p3KqvE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v551a/Hydrus.Network.551a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v551a/Hydrus.Network.551a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v551a/Hydrus.Network.551a.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v551a/Hydrus.Network.551a.-.Linux.-.Executable.tar.gz

I had a great week mostly fixing bugs.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

First off, thanks to a user, you can now turn off the thumbnail 'fade' transitions under options->thumbnails. This will just make any thumbnail appearance or change happen instantly, in one frame.

Secondly, users who are on PyQt (macOS, and some who run from source), had some bad crashing issues last week. I changed one tiny thing, and it turns out PyQt had a big problem with it--Mr Bones and 'open externally' panels and the ratings hover in the media viewer all displayed wrong and caused program instability. Thanks for your reports, sorry for the trouble, and should be all fixed now!

Other than that, I just fixed a bunch of different little things. Some transparent image handling, a weird timezone calculation, database counting issues, and a ton of file move/copy failure handling. Check the changelog for details.

next week

I'd like to do something new, so maybe 'system:has transparency'.
Open

R: 4 / I: 1 / P: 5

Version 548

https://youtube.com/watch?v=Tf8GG3gwam8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v548/Hydrus.Network.548.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v548/Hydrus.Network.548.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v548/Hydrus.Network.548.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v548/Hydrus.Network.548.-.Linux.-.Executable.tar.gz

I had a good week. Thumbnail shortcuts are now customisable.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

First off, I cleared out a ton of bad thumbnail keyboard shortcut code and moved it all to the newer system under file->shortcuts. There is a new shortcut set there, called 'thumbnails', that governs opening the media viewer, the archive/delete filter, selecting files, and moving the thumbnail focus. Everything is now customisable (although I'd stay away from the thumbnail focus stuff unless you really want to wade through a big list), and I have added some new commands for 'select inbox', 'select trash' and similar, for more file selection options. This work clears out some really ancient code, and it should be much easier to extend thumbnail shortcuts in future, including hooking mouse-clicks into the customisable system and adding ctrl+selection logic.

Thanks to a user, we now have renderable krita files! We've got it set like PSDs for now, where you'll get 'open externally' on the preview and the full image in the normal viewer. As always, if you have files that don't work or render crazy, please send them in!

An issue with file export drag-and-drops sometimes giving the wrong file when those files share the same export filenames should be fixed!

Just a small thing, but a couple of places that were unintentionally sorting service lists in a random way now sort them alphabetically, as indended. The F9 new page picker was one of these, so if you have multiple local file domains, you may need to learn some new muscle memory!

next week

With luck I'll have some file storage stuff done, and I'd also like to clear a 'thumbnail fill' rewrite I've been planning.
Open

R: 3 / I: 0 / P: 6

Version 550

https://youtube.com/watch?v=yj2g8l1JqWI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v550/Hydrus.Network.550.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v550/Hydrus.Network.550.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v550/Hydrus.Network.550.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v550/Hydrus.Network.550.-.Linux.-.Executable.tar.gz

🎉 Merry 550 🎉

I had a good week. I fixed some important bugs and reworked 'scale to fill' thumbnails.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I fixed an issue with the login system where, for certain scripts, a job that was supposed to be waiting on an ongoing login process could nonetheless start early. All jobs are more patient about waiting for their specific login process to completely finish, no exceptions.

I added a maintenance system to explicitly close and clear persistent network connections. I had underestimated how aggressive my underlying libraries were at closing these no-longer-used 'keep-alive' network connections, and a half-dead object was occupying your OS's network slots for longer than desired. Network connections now close promptly five minutes after the last use.

I fixed some thumbnail display bugs, and, particularly, 'scale to fill' thumbnails are much more sensible behind the scenes. Previously, the thumbnail on disk was of the cropped visible region (which was a little complicated to figure out and maintain); now, things are simple--it just stores a full-image thumbnail at the appropriate larger scale and then draws the cropped area of that on demand. There are several technical benefits to this, and it saves some CPU in the long run, fixes some bugs for things like blurhash which need the full image to work properly, and it means an external process like the Client API can now ask for a thumbnail and get the full image rather than an arbitrary crop.

If you are set to 'scale to fill', your thumbnails will regenerate as you browse. Fingers crossed, you will not notice any visual difference. Let me know if the regeneration is annoyingly laggy--I didn't want to schedule a ton of regen CPU for you all this update, but I can figure a 'catch up in the background' thing out if it is a problem.

next week

More small work and bug fixing, I hope. If I have time and energy, some file storage overhaul work.
Open

R: 1 / I: 0 / P: 6

Version 549

https://youtube.com/watch?v=VR9oZZcdYNU [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v549/Hydrus.Network.549.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v549/Hydrus.Network.549.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v549/Hydrus.Network.549.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v549/Hydrus.Network.549.-.Linux.-.Executable.tar.gz

I had a good week, improving performance and adding some advanced tools for future testing.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

I fixed some deleted file records in the database. When you update, it'll take a moment to recalculate some things.

The options->maintenance and processing page has a ton of new settings to tune how hard the various background maintenance systems work and rest. These settings are mostly only for debugging, but if you are an advanced user and feel brave, you can play with them. If you have the 'memory explosion on PTR processing' problem, please check the changelog for instructions.

I optimised the taglist sort and update code a bit. Very large pages should be a little less laggy.

If you run from source, the 'easy setup' scripts have a couple new choices for setting up Qt, including choosing to type a specific version in. If you had the issue where your menus were in the wrong position, there is now a new (t)est Qt version (6.6.0) to try out, too.

next week

I got caught up in this 'background' work this week. I'd like to do something more visible and fun next week, but we'll see!
Open

R: 1 / I: 0 / P: 6

Version 547

https://youtube.com/watch?v=Cjkjba-qbhE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v547/Hydrus.Network.547.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v547/Hydrus.Network.547.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v547/Hydrus.Network.547.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v547/Hydrus.Network.547.-.Linux.-.Executable.tar.gz

I had a good week mostly fixing bugs.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Most importantly, I banged my head against a persistent knot of mpv crashiness this week, and I actually got somewhere. A variety of situations where mpv could fail to load a weird file, which would then lead to hydrus program instability, are now much much better, usually no crash worries at all. Also, there's a little more error handling, stuff like a placeholder image to show instead of the broken file, and an error popup to tell you what happened. I cannot promise that this improves mpv on any system that already had trouble just getting it working (e.g. macOS, some Linux), but I think that heart-skipping moment of 'will it crash?' lag when an mpv load fails should be solved. Let me know how you get on!

Thanks to a user, we have support for 'djvu' files, which are basically an open source pdf format.

If you have a client with hundreds of search pages open, please let me know if your session is any less laggy, particularly if you get bumps of lag every five minutes or so. I rewrote some ugly code and think I eliminated some background CPU overhead for your situation.

There's a handful of smaller fixes and tweaks too, nothing super important, but check the changelog if you are interested.

next week

I'm sorry to say my past couple of months have been pretty low productivity overall. Just life stuff cutting at my time and energy, mostly. I'm going to keep my head down and not change anything drastic, but it has been on my mind, and I regret not being able to push on larger projects. We'll see how the new year goes, I think. Anyway, I expect I'll clear out some more jobs like this next week. I've also been dabbling with the idea of a new 'hash sort' to preserve the sort of a 'system:hash' or downloader highlight in various situations, so I might explore that.
Open

R: 1 / I: 0 / P: 6

Version 546

https://youtube.com/watch?v=aFx7woWkZbc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v546/Hydrus.Network.546.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v546/Hydrus.Network.546.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v546/Hydrus.Network.546.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v546/Hydrus.Network.546.-.Linux.-.Executable.tar.gz

I had a simple week. I mostly cleaned code, so there isn't much to talk about except for some bug fixes to recent systems.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

misc

I accidentally screwed up PSD thumbnail colours at the last minute last week while fixing something else. This is fixed, and any borked PSD thumbnails should regenerate themselves soon.

The new file history chart's cancel button works better, and I fixed some of the counting logic in the archive/inbox lines when you have a search filtering the results. Let me know if you still see archive numbers that are way too high.

If you have run into the new 'blurhash' thumbnails and don't like them, hit the new checkbox in options->thumbnails to turn them off!

next week

I really got lost in code refactoring this week. It is always worthwhile work to do, reshaping old bad ideas and preparing for the future, but it is unexciting, and with a janked out codebase like mine it often feels like bailing out the ocean.

For next week, I'd like to get back to my file storage system improvements, ideally getting background migration ready to go.
Open

R: 1 / I: 0 / P: 6

Version 545

https://youtube.com/watch?v=OLJhFROQg5A [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v545/Hydrus.Network.545.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v545/Hydrus.Network.545.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v545/Hydrus.Network.545.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v545/Hydrus.Network.545.-.Linux.-.Executable.tar.gz

I had a good week. I did some small things, and a user contributed some cool things!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

blurhash

tl;dr: we have some special blurry thumbnails now, but you won't see them much yet.

Thanks to a user who did a bunch of research and work to get this going, the client now generates the 'blurhash' of all files that have a thumbnail. This hash is essentially a micro-thumbnail, only about 34 bytes, that shows a very blurry coloured impression of the general shape of the image or video. They are usually used as placeholders when it may take time to load something--you have probably seen something similar on slower news websites.

You won't see these in hydrus itself much, since thumbnails load fast enough that we don't have to worry about placeholders. In the advanced ways of seeing files you don't actually own, however, I will now show you any file's known blurhash rather than the default 'hydrus' thumb. Same deal for damaged/missing files--if I can't fetch a thumb, I'll now fall back to the blurhash.

The more important thing is these hashes are now available on the Client API, under the normal 'file_metadata' call. If you are implementing a browser or similar and have access to a fast blurhash library, try them out! I've scheduled all users to generate the blurhashes for their existing files in the background, which will take a few weeks/months for users with hundreds of thousands of files (although, if you are working with this and want to hurry it along, remember the queue is manageable under database->file maintenance.

other highlights

The file history chart (help->view file history) now has a search panel, just like Mr Bones! You can search your import, archive, and delete history for creator x, filetype y, or any other query you can think of. Some of the math here got a bit weird, and I am sure I have missed out several 'cannot provide good numbers for this domain' situations, so let me know if you get any really stupid results or outright errors. There's more work to do here, like a button to hide the search panel, which I hope to push on in the near future.

Thanks to the same user above, we also have epub support! No 'num_words' yet, but it turns out epubs are just zip files with some html, so I think it'll be doable in future. Also, some rare incorrect jpeg rotations (for 'MPO' jpeg files) are fixed.

If you right-click on a selection of files, the 'open->similar files' menu now has a 'in a new duplicate filter page' command. This will initialise the filter page just looking at those files, hopefully making it simple to clear out the potential duplicates in any particular selection.

Unfortunately, I am retiring the Deviant Art artist search and login script. DA have been slowly killing their nice old API, and the artist search just went. Individual page URLs still seem to work, but I suspect they will be gone soon. Gallery-dl have a nice DA solution that works with the new API, so if you can't find the same content on a more open booru, that's my best recommendation for now.

next week

I want to take a very simple cleanup week. Nothing too exciting on the changelog front, but I'll refactor some of the worst code into something nicer to work with.
Open

R: 5 / I: 0 / P: 6

Version 544

https://youtube.com/watch?v=T9IjJIX50Ls [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v544/Hydrus.Network.544.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v544/Hydrus.Network.544.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v544/Hydrus.Network.544.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v544/Hydrus.Network.544.-.Linux.-.Executable.tar.gz

I had a good week. File storage locations can now have max size limits. There is a security fix in this release, and users who run from source will want to rebuild their venvs this week.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

max size

This is just for advanced users for now.

So, in migrate database, if you have multiple locations set for your files, you can now, for all but one, set a 'max size', limiting the total mass of files it will store. The idea here is that if you have a 2TB drive and a 500GB one, then rather than having to keep playing with the weights to ensure the 500GB doesn't get too full, you can now set the 500GB location with a 450GB limit and hydrus will allocate that space and no more.

There are a couple of limitations. It isn't perfectly precise, so if your space is tight, give it a little padding (although remember that drives should be at least 10% free space anyway!). Also, it won't yet enforce these rules outside of the 'move files now' button. If you set a tight limit and then import many files, it will blow over that limit (at least until you open up migrate database and hit 'move files now' again to rebalance).

The next step here is to implement automatic background file migration for always-on checking so we can have these rules apply without regular user intervention. My aim is to get rid of the lagtastic 'move files now' entirely so no file migration blocks other use. I hope to have this done in the next few weeks.

webp

A remote execution (very bad) vulnerability was recently discovered in the main webp library. You probably noticed your chrome/firefox updated this week, which was fixing it. Our 'Pillow' image-loading library uses libwebp, and they rolled out an update themselves this week. The builds today incorporate the fix, so if you use them, just update as normal and you are good. If you run from source, reinstall your venv.

However, if you are Windows 7 (or otherwise on Python <=3.7), I am afraid you cannot get the fix! As I understand, it just won't work for you. There is now an additional choice in the 'advanced' setup_venv script to choose the older Pillow version, so you can keep running, but I'm afraid with this and other things you lads are now solidly in the phase of limited security updates. I will keep supporting you as long as it is generally reasonable, but if you still want to use Win 7, I think the train is running out of track.

misc

When images fail to render, they now show something proper in the media viewer. (previously they were just blank forever and made some error popups).

After last week's updates, Mr Bones gets a readability/layout pass.

The client should stop mistaking various text files for mpegs in the pre-import phase!

I think the Docker client wouldn't boot last week, same for anyone else without the PSD-tools library. Sorry for the trouble, should be fixed now!

next week

I'd like to add a file search to the file history chart and get stuck into more file storage infrastructure updates.
Open

R: 1 / I: 0 / P: 6

Version 543

https://youtube.com/watch?v=3zw3sRLVPN0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v543/Hydrus.Network.543.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v543/Hydrus.Network.543.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v543/Hydrus.Network.543.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v543/Hydrus.Network.543.-.Linux.-.Executable.tar.gz

I had a good week. Mr Bones gets an update. If you run from source, you might like to rebuild your venv today to get some new parsing support.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

Mr Bones

So, the statistics under help->how boned am I? now have an attached file search. The numbers will change based on whatever you search, so you can now see the average size of your pngs, the archive/delete ratio for creator x, how many times you have viewed pictures of sonic the hedgehog, or any other search you can think up. The default search is what the dialog was fixed to before, 'system:everything' on 'all my files'.

This turned out to be surprisingly complicated, and to stop it spiralling in complexity, I culled some of the odder scenarios. If you do a multiple-file-domain search, you won't get 'deleted' or 'earliest import' numbers. And if you search 'deleted from x', you are going to get some jank logic. EDIT: I also disabled the potential dupes count right before release, it was performing awfully on my large IRL database. Keep it simple, and let me know what turns up. I love these sorts of weird statistics.

I'd like to do the same for the file history chart soon. Should be a bit easier!

misc

PSDs should render more quickly and with less memory!

PDFs no longer spam the log on thumbnail generation!

Bitmaps files (.bmp) are now fully supported. Since the program started, these have been converted to png on import, mostly because the bmp format personally annoys me. This was always in conflict with the tenet that hydrus not alter files in any way, and I think it was ultimately a mistake. Bitmaps now stay as-is, and we've happily untied a couple of awkward knots in the code as a result.

If you are a downloader maker, there's a new 'datestring to timestamp (easy)' String Converter step. It uses an excellent external library ('dateparser') that can parse pretty much any datestring you can think of, including stuff like '2 hours ago'. If you run from source, this is the thing you are rebuilding for today. I have left all the existing 'datestring to timestamp' conversion steps in place for now, as '(advanced)', but if this new step integrates well, I think it'll supercede them and I may ultimately convert them all over.

next week

I'm going to have a busy week IRL, so work might be limited. I trudged through more file storage overhaul this week, and while I didn't finish the 'max num bytes' option as I had hoped, it is all ready behind the scenes. I just need to put some UI together and I think we'll have it working.
Open

R: 3 / I: 0 / P: 6

Version 406

https://youtube.com/watch?v=UZr6b4mBoxI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v406.tar.gz

I had a good week. It is now easy to deduplicate queries in subscriptions.

subscriptions

Managing multiple subscriptions can get complicated. Figuring out which queries are in which sub, whether and where you have duplicates, is tedious. A recent bug with query pasting also could have introduced some duplicates within the same sub. This week should make it simple to manage.

The main manage subscriptions dialog now has a 'deduplicate' button. It lights up any time your subs of a particular downloader have the same query, whether that includes dupes within the same sub or across different subs. Click it, and it launches a thorough process to determine what you want to do:

- First, it asks if you want to deduplicate by case or not--e.g. whether "Samus_Aran" and "samus_aran" should be considered dupes, or only exact matches.
- Then it summarises which downloaders appear to have dupes, letting you choose which to work on.
- Then you have the option of selecting which of the queries you wish to deduplicate--e.g. if you want to do two runs to dedupe, say, artists to one sub and characters to another.
- Then you select which subscription(s) will retain the deduplicated queries.

Every step explains itself. Mostly it is simple, but there are some odd situations. When a sub dedupes queries within itself, it keeps the one(s) with the most files. I hope I have accounted for everything reasonable.

Adding to caseless matching, also added is a 'lowercase' button, which will coerce all the queries in the selected subscriptions to lowercase, just to help keep things neat. And lastly, when you paste queries into a sub, the 'already in the sub' test is now caseless.
Open

R: 3 / I: 0 / P: 6

Version 534

https://youtube.com/watch?v=wpUEdGDdaTQ [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v534/Hydrus.Network.534.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v534/Hydrus.Network.534.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v534/Hydrus.Network.534.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v534/Hydrus.Network.534.-.Linux.-.Executable.tar.gz

I had a good week. There's lots of smaller fixes and improvements, and ratings are added to the Client API.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Thanks to a user submission, hydrus now recognises and supports importing .sai2 files! Basic metadata for now, but I'll see about resolution and thumbnail support in future.

It turns out PNG and WEBP files can have EXIF data, and our EXIF scanner works on them! All newly importing PNGs and WEBPs will be scanned, and on update, you'll have a yes/no dialog asking if you want to scan your existing PNGs. If you have a gigantic database, or your files are on a remote cloud storage, you might want to defer it, but it isn't really a big deal.

I banged my head against the media viewer 'notes hover' layout code, and I was actually really successful this time. Many of the borked layout issues are fixed, and I think I've completely fixed the overlapping hover problems in the duplicate filter.

I fixed a bunch of weird bitmap paste problems in the new 'similar files' paste button. Let me know if you still have any issues!

Just a small thing, but the 'loading' page in hydrus, when you refresh a search, is no longer grey. It is the same colour as the normal thumbgrid, so the loading flicker is gone.

Twitter is gone from the downloader defaults for all new users. All existing users should look to migrate away, too. I hope to roll out some downloader engine improvements in the near future that'll make nitter (on multiple domains) easier to set up, but there aren't a lot of excellent solutions.

I also added ratings to the Client API. The service object now talks about min/max stars and star shape, the file metadata call gives every file's ratings, and you can set and clear ratings too with a new command and permission. It is all fairly simple, but there's several possible data types depending on the rating type, so check out the Client API help for details and examples.

next week

I regret that I again did not find time for PTR janitor workflow improvements this week. Now that ratings on the Client API is clear, this is top priority. Depending on how things go, I may take a two-week release, since I definitely want some solid work done here.

R: 2 / I: 0 / P: 6

No discord hydrus shenanigans for now.

Locked out of discord because of their shitty mobile verification, so doing my part tagging paywalled furry art ¯\_(ツ)_/¯

> https://youtube.com/watch?v=SMTz9nIUkGc [Embed]

R: 3 / I: 0 / P: 7

unable to keep my tags visible and overall finicky

whattup,

I've been fiddling with hydrus for a bit but cannot figure out how to keep my tags visible and usable at all times. I have two tags so far, but expect to use dozens, if not hundreds in total in the future.

The whole thing isn't easy for me introductory tutorial that I've gone over so far has basically been "select image, press f3, do ur tag stuff".

Can I see all my own created tags without selecting an image and pressing f3 and be able to edit/delete them? Also, how do I keep tags in the "selection tags" section at all times?

Thanks in advance.
Open

R: 4 / I: 0 / P: 7

Version 451

https://youtube.com/watch?v=Cv87NJ2xX0k [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v451/Hydrus.Network.451.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v451/Hydrus.Network.451.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v451/Hydrus.Network.451.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v451/Hydrus.Network.451.-.Linux.-.Executable.tar.gz

I had a great week cleaning code and fixing bugs. If you have a big database, it will take a minute to update this week.

all misc this week, most bug fixes

I fixed a critical bug in tag siblings. It was causing some tag siblings to be forgotten, particularly on local tag services, and particularly when a tag had a sibling removed and added (e.g. 'delete A->B, then add A->C') in the same transaction. I will keep working here and will trigger a new sibling reprocess in the future just as 450 did so we can fix more PTR issues.

The new content-based processing tracking had a couple more issues. Some Linux users, in particular, it just broke for, due to a SQLite version issue. I have fixed that, and I have fixed some issues it caused for IPFS. There are new unit tests to make sure this won't happen again.

I fixed an issue with sessions recently not saving thumbnail order correctly!

I fixed issues with some of the new Client API file search parameters not working right!

Big video files should import a bit faster, and will show more status updates as they do their work.

We have had more anti-virus problems in recent weeks. We'd hoped building on github would eliminate them, but it hasn't completely. A user helped me trace one issue to the Windows installer. We have 'corrected' it (believe it or not, it was removing the 'open help' checkbox on the final page of the install wizard, and yes the reasons for why this was causing problems are ridiculous), so with luck there will be fewer problems. Thank you for the reports, and let me know if you have more trouble!

full list

- stupid anti-virus thing:
- we have had several more anti-virus false positives just recently. we discovered that at least one testbed used by these companies was testing the 'open html help' checkbox in the installer, which then launched Edge on the testbed, and then launched the Windows Update process for Edge and Skype, which was somehow interacting with UAC and thus considered suspicious activity owned by the hydrus installer process, lmao. thereafter, it seems the installer exe's DNS requests were somehow being cross-connected with the client.exe scan as that was identified as connected with the installer. taking that checkbox out as a test produced a much cleaner scan. there is a limit to how much of this nonsense I will accomodate, but this week we are trying a release without that 'open help' link in the installer, let's see how it goes
- semi-related, I brushed up the install path message in the installer and clarified help->help will open the help in the first-start welcome popup message
Open

R: 1 / I: 0 / P: 7

Version 542

https://youtube.com/watch?v=vDPoR5PHw7Y [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v542/Hydrus.Network.542.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v542/Hydrus.Network.542.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v542/Hydrus.Network.542.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v542/Hydrus.Network.542.-.Linux.-.Executable.tar.gz

I had a good week. There are improvements to PDFs and system predicate parsing.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Thanks to a user, PDFs now have thumbnails! I'm happy and surprised with how this turned out. The module it uses also offers raw text access, so I have rejuvenated a I-think-good-enough 'num words' count for PDFs. Also, PDFs can have a little embedded metadata like Author and Title, and this is viewable on the special button at the top of the media viewer. On update, all your existing PDFs will be scheduled for some file maintenance work to figure all this out, so give it a little while to catch everything up. As always, please send in any PDFs that get weird results. I expect we'll do a little more here, particularly pulling the embedded data into hydrus proper as tags etc...

System predicate parsing (e.g. by typing 'system:inbox' into the tag autocomplete) has several improvements. All comma-grouped numbers are now parseable, so you can paste 'system:width = 1,920' and it'll work. 'system:tag as number' has some special fixes, and several tags have presentation updates within the client to make sure that the text they are producing for a label is actually something that parses back to them.

Tag export sidecars now have a 'tag display type' selector to choose between 'storage' and 'display' tags. Until now, all the sidecars were all 'storage', meaning they did not include parent or sibling tags (like the list you edit in manage tags), whereas 'display' includes sibling replacements and implied parents (like the normal file search view shows). Existing tag export sidecars will stay on 'storage', but new ones will default to 'display'.

next week

I did some more file storage updates this week, but it remained pretty boring behind the scenes stuff. I'm going to keep bashing away, and I'd love to have a bit of other fun too, maybe custom searches for Mr Bones and the file history chart, which I have been meaning to do for ages.
Open

R: 1 / I: 0 / P: 7

Version 541

https://youtube.com/watch?v=xJRAubWCkcc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v541/Hydrus.Network.541.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v541/Hydrus.Network.541.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v541/Hydrus.Network.541.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v541/Hydrus.Network.541.-.Linux.-.Executable.tar.gz

I had a good week. There's a mix of all sorts of work.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Thanks to a user, we are adding support for two more filetypes today: QOI, which is a png alternative, and Procreate, an Apple image project file. QOIs have full support, Procreate files just have thumbnails.

If you use one of the darkmode styles on Windows and your tooltips suddenly got too dark to read a couple weeks ago, please check out the new 'alternate-tooltip-colour' variants of the styles in the options. It should fix you up for now. Try switching back when we move up to a new version of Qt again--it is a bug on their end.

There's two new checkboxes in options->media to ignore 'uninteresting' import and modified times. By default, if the modified time is very close to when a file was added to 'my files', I hide it in the media viewer. If you would prefer to always see it, you can now set this.

Importing massive apngs is now significantly faster.

The parsing system has a new 'String Joiner', which does string concatenation. It can glue the strings together with any custom text, including the empty string for pure concatenation, and it can join groups in 1-2 1-2 1-2 or 1-2-3 1-2-3 style patterns too.

The Client API can now fetch the siblings and parents of tags. If you want to get into siblings and parents but have never done it before, it can get complicated, so brace for impact.

file storage upgrades

I started on this work this week, but it is all boring behind the scenes stuff so far. I'm happy with the progress though, and I'd like, in the nearish future, to have:

- lower file access latency when you have millions of files
- the ability to say 'store no more than 200GB in this folder'
- background migration of files rather than the current block-the-whole-program-for-ages 'move files now' system

I'll keep chipping away at this over the next few weeks. I feel good about it.

next week

More like this. Catchup on older issues and pushing on file storage.
Open

R: 3 / I: 0 / P: 7

Version 540

https://youtube.com/watch?v=ANDQoEl2StM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v540/Hydrus.Network.540.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v540/Hydrus.Network.540.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v540/Hydrus.Network.540.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v540/Hydrus.Network.540.-.Linux.-.Executable.tar.gz

I had an ok week. There's a mix of smaller work.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

system:filetype parsing now supports more human-friendly words. You can type 'system:filetype is image' or 'system:filetype is static gif, png' and it should just work. I think that any existing filetype predicate string (e.g. if you right-click->copy it) should now parse ok.

I fixed a bunch of broken 'system:similar files' image bitmap pastes that were not producing the correct pixel hash. Also, I've started a background job today to fix some lingering bad duplicate pixel data for pngs, so if you have recently done a duplicate filter with 'must not be pixel dupes' but got some anyway, I hope this will fix itself soon.

I added links to https://github.com/Garbevoir/wd-e621-hydrus-tagger (and https://github.com/abtalerico/wd-hydrus-tagger, which it is based on) to the Client API help. These use the recent 'I give text, it draws an image' AI tech everyone has been messing around with to do the reverse--you give it an image, and it automatically tags it. This stuff is just getting ready to seriously play with, so if you are interested, give it a go!

next week

I didn't get to the file storage upgrades I wanted this week, so I'll give it another go.
Open

R: 1 / I: 0 / P: 7

Version 539

https://youtube.com/watch?v=htZEYN1TkOE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v539a/Hydrus.Network.539a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v539a/Hydrus.Network.539a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v539a/Hydrus.Network.539a.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v539a/Hydrus.Network.539a.-.Linux.-.Executable.tar.gz

I had an ok week. There's full PSD support and a variety of quality of life improvements.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

PSD files

Just like last week, if you run from source, you might like to rebuild your venv again this week, and you'll get the PSD support. Just run 'setup_venv' again.

Thanks to a user, PSD files should now show in the program just like any other image! Since it takes a little extra CPU to render them, we're starting with conservative view settings--PSD files will show in full in the normal media viewer, but you'll get the cheaper 'open externally' button in the preview panel. If you want to try showing them everywhere, you can change the view settings under options->media.

PSD thumbs are also much nicer now, and your existing PSD thumbnails should regenerate very soon in the background.

As always, if you run across a PSD that has crazy colours or bad transparency--or if it straight up won't load at all, which we think some will--please send them in and I'll see what I can do.

other highlights

I'm splitting 'gif' files into 'static gif' (under images) and 'animated gif' (under animations) this week. It isn't a huge thing, and there aren't all that many static (i.e. non-animated) gifs going around anyway, but it simplifies some view settings as well as some code, and it aligns with how the other formats like png/apng work.

Rating system predicates are now parseable. Type 'system:has rating (service_name)' or 'system:rating for (service name) > 4/5' into the tag autocomplete, and it should prefill the result. It should work for all like/disklike, numerical, and inc/dec rating services, and like the other parseable system predicates, this extends to the Client API. Try it out!

I fixed the duplicate filter's right-hand hover window back in place. It was a mistake to have it 'avoid' the notes hover, since the buttons bounced around too much for easy clicking, so now they can just overlap, with the duplicates hover taking precedence.

next week

I'm due some larger work, and I think I'm going to tackle some file storage improvements. The migrate database dialog sucks, and the file storage system in general groans when it has millions of files, so I'd like to have the first steps towards user friendly storage settings and cleverer storage overall. We'll see!
Open

R: 1 / I: 0 / P: 7

Version 538

https://youtube.com/watch?v=8yXk3_o1Hoo [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v538/Hydrus.Network.538.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v538/Hydrus.Network.538.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v538/Hydrus.Network.538.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v538/Hydrus.Network.538.-.Linux.-.Executable.tar.gz

I had a great week. There's some new libraries and new file support.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

notes

If you get a popup on update about missing indices, don't worry. Something I forgot last week is being fixed. Just let it work.

If you run from source, I recommend you rebuild your venv this week. Just run the 'setup_venv' script again with the same settings you chose before. You'll get new Qt, OpenCV, and the HEIF support. If you are on an older OS and run into trouble with Qt, the setup now has a 'Qt (o)lder' option that gives you something we know is very stable and compatible.

=highlights

The main UI and image libraries that hydrus uses are updated today. We've been in an awkward place with Qt for a while, stuck on a version I regret updating to, so I hope this jump relieves this. It should speed some image load, reduce some flicker, and fix some UI scaling issues. There are no special install instructions, and we've been testing for months on different computers and haven't seen any specific update problems, so I'm not expecting any significant issues. If you do run into anything, then please roll back to your backup and let me know.

Thanks to a user, we now have full support for HEIF, HEIC, and AVIF files! They import and work like any other image. We also support the 'sequence' version of these files, which are essentially like (currently super rare) high quality animated gifs. I'm very new to all these formats, and they are obviously less common, so if you have a weird version or your colours are all messed up, I'll be interested in examples. I don't even have a 'HEIC sequence' to test with!

My deferred delete system is complete. Most users don't have to ever think about it, but it is under database->database maintenance. There's an UI panel to track what still needs to be deleted and some basic settings to control when it does its work. If you delete the PTR or do a large regeneration job, the (very slow) actual delete of the existing data now happens in pieces in the background in the future. I'm really pleased with the performance, and even deleting 30-million-row tables seems to work very well, but if you delete the PTR (2 billiow row table!) and get unacceptable lag every few seconds, check that menu and disable the work during idle/normal time and let me know, and we'll figure it out.

tags->migrate tags has cleverer file filtering. You can now select the tags for files 'deleted from x' or 'currently in y or z' real easy.

next week

I'm generally happy with my current productivity, but I'm falling behind on many things at once. I would like to focus on system predicate parsing and Client API for a week, because it keeps slipping.
Open

R: 1 / I: 0 / P: 7

Version 537

https://youtube.com/watch?v=pg-yeXEvVwA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v537/Hydrus.Network.537.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v537/Hydrus.Network.537.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v537/Hydrus.Network.537.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v537/Hydrus.Network.537.-.Linux.-.Executable.tar.gz

I had an ok week. There's a mix of bug fixes and a couple of neat improvements.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

First off, the filetype selector, which you'll see in 'system:filetype' and any File Import Options, is completely rewritten to a checkbox tree. It is easier to work with than the old expanding panels system and scrolls smoothly when it gets internally tall. I've been worried about adding more filetype support for a while because of how busy this widget was getting, but this will relieve things.

Also, the 'applications' group is now split up. All the archives like zip and rar are now in 'archives', and all the xcf, psd, and krita formats are in 'image project files'. It doesn't matter much yet, but these groups also exist as separate rows in the viewing settings under _options->media_.

Secondly, I have figured out a 'fast delete' method for very large database objects. Deleting the PTR is now an instant operation! Unfortunately, under the hood this is really just a deferred delete, so the object still uses its hard drive space until it is deleted later in the background in pieces. Unfortunately x2 is that I haven't written the maintenance job that does the delete yet, lol, so if you need to delete some service or do a large database regenerate job, please bear with the bloat for a week.

If you updated last week, jumping up several versions at once, and got a popup that some EXIF scanning update did not work, don't worry. I fixed the problem, and nothing important was actually broken anyway. Thanks for the reports!

I fixed the grid layout on tags->migrate tags, which exploded under my nose. Sorry if you were recently confused by this!

next week

I'll finish this background deletion maintenance system and keep on trucking on smaller fixes. I've got a lot of immediate work like this to do and github issues to catch up on. I'd also love to get some more system predicate parsing, for ratings and more, working.
Open

R: 1 / I: 0 / P: 7

Version 536

https://youtube.com/watch?v=EtVjGWpbE7k [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v536/Hydrus.Network.536.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v536/Hydrus.Network.536.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v536/Hydrus.Network.536.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v536/Hydrus.Network.536.-.Linux.-.Executable.tar.gz

I had a mixed week, but I fixed some things, and there is more filetype support. There are a couple of database fixes this week, so anyone with a large database, including anyone who syncs or has synced with the PTR, can expect the update to take several minutes.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Thanks to a user, the program now supports gzip (a type of archive) and xcf (GIMP savefiles)! SVG thumbs also look correct now if you have 'fill' thumbnails. Since the 'filetype selector' widget is getting pretty stacked, I'm going to rewrite it very soon, and I think we are going to have one or more groups, maybe spitting raster/vector and grouping 'image' application types like psd, krita, and xcf, together.

I fixed an issue where even if you deleted a tag from all the files somewhere (i.e. reduced its autocomplete count to 0), it wasn't disappearing from autocomplete results. It shouldn't happen again, and one of the updates this week will be scanning through your autocomplete data to remove all existing orphans.

Otherwise it is mostly background work this week! One thing is more responsive, another is more efficient, and some UI protects against mistakes, but none of it is too interesting. There's a clever delete-lock update to the Client API if you handle a lot of advanced file deletions.

future build

There's another test build out today, here: https://github.com/hydrusnetwork/hydrus/releases/tag/v536-future-2

It is the same 536 but uses newer libraries. If you have trouble with the current version of Qt (e.g. borked high UI scale text sizes) or just want to test out newer tech, feel free to have a go. We've had some real problems with the recent crop of Qt updates, but this one looks tentatively good, so I'm very hopeful this jump will put the issues to bed. Let me know if there are any problems!

next week

I'm behind on my github issues, so I'll give them a good look, and otherwise continue pushing at my immediate backlog of smaller jobs.
Open

R: 2 / I: 0 / P: 7

Version 535

https://youtube.com/watch?v=qv6yDjtIl8k [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v535/Hydrus.Network.535.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v535/Hydrus.Network.535.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v535/Hydrus.Network.535.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v535/Hydrus.Network.535.-.Linux.-.Executable.tar.gz

I had a good couple weeks working on better petition-processing workflow for repository janitors. Most users won't see this, so you can ignore most of this release!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

krita and SVG

Thanks to two different users, hydrus gets Krita (.kra, .krz) and SVG (.svg) support today. Both formats also get thumbnails!

These formats can get quite complicated, so if you have one that won't import right (for Krita, it'll look like a zip; for SVG, probably HTML), please send it in and I'll have a look. Same for if the thumbnails are broken, although there's a limit to how clever we can go there.

Adding SVG also exposed that I don't have a good category/pipeline for handling vector vs raster graphics. Several things like thumbnail size calculations are different with an SVG (which has no true fixed resolution), so I will be doing several waves of background updates in future to handle this and other vectors better. Who knows, we might have beautifully zooming SVG media viewer support with one day.

petitions

Ok, this was almost all of my work this past two weeks. It kind of kicked my ass, but I'm happy it is done. It changes UI that only a handful of users see, so most users can skip over it completely.

For server admins, this new tech needs a server update, and it is not compatible with older clients (nor can newer clients talk to older servers), so please plan to update your server and janitors' clients at roughly the same time. It isn't going to break anything, but the client won't be able to do janitor work and will produce some harmless errors either way.

For janitors, I've made the petition processing page handle multiple petitions at once. There's a new panel, a multi-column list, and when you click the 'fetch petitions' button, it populates with as many petitions as you like. You can see and sort by the action (ADD, DELETE, REPLACE), account id, reason, and total size (e.g. '60 mappings in 12 petitions'), and then double-click any to highlight it in the normal petition UI you are used to.

This is about speed of processing. When you process a petition, it starts uploading in the background and the page moves immediately on to the next one. No more waiting for the server to summon another petition. Also, it gives you a better overview of your queue and even lets you mass-approve/deny a set of petitions when one account or reason keeps making good or bad stuff. You can fire off twenty petitions in one go and quickly move on to the next.
Open

R: 1 / I: 0 / P: 8

Version 533

https://youtube.com/watch?v=x33quKIyVu0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v533/Hydrus.Network.533.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v533/Hydrus.Network.533.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v533/Hydrus.Network.533.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v533/Hydrus.Network.533.-.Linux.-.Executable.tar.gz

I had an ok week. There's a mix of different work and a fix to the recent macOS crashing.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

all misc this week

Subscription popup windows should update far more smoothly and frequently this week. Some of the 'should I check for new text?' logic was broken when I embedded the popup toaster into the main UI a few months ago--it is fixed.

If you right-click an image that is larger than 1,024 pixels on a side, the share->copy menu now has a second 'copy bitmap' that crops the image down for easier and faster pasting in lookup services like SauceNAO.

I added some info about the not-well-documented tag/predicate mouse shortcuts (e.g. shift+double-click to edit system predicates) to the help here: https://hydrusnetwork.github.io/hydrus/getting_started_searching.html#editing_predicates

I added a link to the Client API help for a neat new tool that finds video duplicates and queues them up in the duplicate filter. Here: https://github.com/appleappleapplenanner/hydrus-video-deduplicator

Since v529, at least some users of the macOS app have had terrible crashing. Last week's attempt didn't do it, but I think we've fixed it now! It was something I never even knew about updating in the background to an incompatible version. Thank you for your patience and help testing, and let me know if you still have any trouble!

If you are an advanced downloader maker, check out the changelog regarding sub-gallery urls. Nothing huge has changed, but they'll inherit and parse tags and some other stuff from their parent parsing step now.

The experimental QtMediaPlayer now initialises with and updates according to the volume and mute settings. Previously it was stuck on 100% always. The volume bar is still hidden on the actual player, but we are moving forward.

next week

I failed to do as I planned this week. I'd hoped to set aside a good whack of time to figure out a PTR tag filtering system, but it didn't happen. I'll see if I can get the time and energy together again this week, and if not, I'll try and kick out some ratings in the Client API instead.
Open

R: 1 / I: 0 / P: 8

https://youtube.com/watch?v=eclMFa0mD1c [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v532a/Hydrus.Network.532a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v532a/Hydrus.Network.532a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v532a/Hydrus.Network.532a.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v532a/Hydrus.Network.532a.-.Linux.-.Executable.tar.gz

The 532a is a hotfix, I messed something up with the new pixiv parser first time around.

I had a good week. I've improved stability and added some little features.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

stability

Some damaged videos were causing mpv to dump out in a way Qt was unhappy with, and it lead to black video screens that would crash after a few clicks. I have written a new error-catching system that should recognise these specific files and immediately unload them along with a warning to the user. Also, all other general errors (like missing metadata or chapter marks, which are common for our sort of files) are now printed to your log for future investigation. As we learn more, I expect to add more 'unload this immediately' and 'don't print to log, this error is nbd' hooks here in future.

The macOS release has unfortunately been unstable for some/all users since v530. Clicking on lists can simply not work and then suddenly cause a crashe. With some user help, I have learned an important potential crash in Qt and tightened up all the respective areas in hydrus code. If you are a macOS user and have had trouble, let me know how you get on with today's release. If you still get crashes from simple list interaction (e.g. in manage subscriptions), please revert to v529, which does not appear to have the problem. There have been no big database changes since then, so all you have to do is re-install v529 and you should be good to go.

The above macOS error handling improvements may have fixed some other unexplained crashes. What once caused a crash should now appear as a normal popup error. Let me know what you see.

other highlights

When you hit one of the 'open these files in a new page' commands, the new page is now search-enabled and starts with a 'system:hashes' of the files it opened with. This lets you quickly add more search predicates to narrow down the selection!

When you right-click a tag, the 'search' menu now lets you open the tag(s) in a new duplicate filter page!

I updated the pixiv downloader again. A couple weeks ago, Pixiv decided that if you aren't logged in, you can't get the 'original' quality of an image. I updated the downloader before my vacation by telling it to get the resized version, but as this was importing low quality files silently, it was not a great idea. I have given it another go, and now, if you aren't logged in to pixiv, it sets those files as 'ignored' and the 'note' on the file log mentions you need to be logged in to get the original quality. If you want the previous 'get the medium quality' version, it is now selectable as a different parser for 'pixiv file page api' under network->downloader components->manage url class links.

next week

I'd like to sink my teeth into a medium size job. It'd be nice to finally catch up on some serverside tech for more PTR updates.
Open

R: 1 / I: 0 / P: 8

Version 531

https://youtube.com/watch?v=7ACn4nCY4lQ [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v531/Hydrus.Network.531.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v531/Hydrus.Network.531.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v531/Hydrus.Network.531.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v531/Hydrus.Network.531.-.Linux.-.Executable.tar.gz

NOTICE! For everyone but macOS, Version 527 had special update instructions. If you are updating from 526 or earlier, please consult the post here: https://github.com/hydrusnetwork/hydrus/releases/tag/v527

I had a good week working on simple fixes and cleanup before my break.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

I also made another future build for v531 here: https://github.com/hydrusnetwork/hydrus/releases/tag/v531-future-1

all misc this week

I fixed editing favourite searches, which I accidentally broke last week!

If you right-click a tag or a file and it has a billion siblings/parents/children/known URLs, the menus now truncate the giga list. Previously, child-heavy tags like pokemon would make a menu that would laggily take up your entire screen. Now it lists about ten items and says '438 more...'.

I fixed the Pixiv parser for guest logins, but it no longer gets the 'original' quality. It seems like they hide this for non-logged-in users now. If you do a lot of Pixiv downloading, figure out a login solution with Hydrus Companion or cookies.txt imports.

If you have some downloaders that were inexplicably halted on 'waiting for a work slot', even after client restart, check them again this week--I cleaned up some bad status reporting.

The Client API gets an overhaul to service info this week. '/get_services' and '/get_files/file_metadata' are affected. If you do Client API work, please check the changelog and the updated help here https://hydrusnetwork.github.io/hydrus/developer_api.html#services_object . The old method of giving service names and types still works, so nothing breaks today, but it is now deprecated and will be deleted in early 2024. Please plan to move over!

next week

I am now on vacation! I'm off for a week to shitpost not-E3. I'll be back at it in a week, and I want to start pushing on my larger projects again. v532 should be out on the 21st. Thanks for the support everyone!
Open

R: 3 / I: 0 / P: 8

Version 530

https://youtube.com/watch?v=Gp1dzdFmwmc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v530/Hydrus.Network.530.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v530/Hydrus.Network.530.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v530/Hydrus.Network.530.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v530/Hydrus.Network.530.-.Linux.-.Executable.tar.gz

NOTICE! For everyone but macOS, Version 527 had special update instructions. If you are updating from 526 or earlier, please consult the post here: https://github.com/hydrusnetwork/hydrus/releases/tag/v527

I had a good week. You can now type system predicates in a normal search, and highlighting big downloaders is nicer.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

The tag autocomplete text inputs on search pages get a couple of neat improvements this week:

First off, you can now type most system predicates, and they'll appear in the dropdown. You type the whole complete predicate, like 'system:imported before 2021-03-09'. This is the same system predicate parser that has been available in some advanced places, now for everyone. Note that it can't do every predicate yet, and while I try to cover several grammatical variants, some it is still quite inflexible. Let me know how you get on, and I'll keep working on it!

Second, the autocomplete now has a paste button. It accepts multiple newline-separated tags and will understand -negation, wildc*rds, namespace:* wildcards, and the new system predicates.

For downloaders, I made the 'highlighting' when you double-click a downloader or thread watcher asynchronous--it now loads the files in the background, only presenting them when the results are ready. This means if you highlight a big 6,000 file downloader, it won't hang the UI for several seconds. You can cancel a load by un-highlighting or highlighting something else, and a file-load that takes more than a few seconds will make a popup with its progress.

next week

I've just got one more week before my vacation, so I'll just do some code cleaning and little fixes again. Thanks everyone!
Open

R: 2 / I: 0 / P: 8

Version 529

https://youtube.com/watch?v=zIY17xqTzmk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v529/Hydrus.Network.529.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v529/Hydrus.Network.529.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v529/Hydrus.Network.529.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v529/Hydrus.Network.529.-.Linux.-.Executable.tar.gz

NOTICE! For everyone but macOS, Version 527 had special update instructions. If you are updating from 526 or earlier, please consult the post here: https://github.com/hydrusnetwork/hydrus/releases/tag/v527

I had a good week. There's a new quick lookup system that lets you search for a file's source without importing it.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

sauce

Every now and then, I am looking at a file outside of hydrus and I can't remember who the artist/character is, or I want to know if I have the artist subbed. When it isn't something the common online source platforms support, I usually download the file, import it to my client, and then do a 'open->similar looking files' on it to see everything I have that looks like it to get more info. I'm basically doing SauceNAO but on my own client. I wanted a way to do this faster.

So, this week, check out the renamed 'system:similar files' predicate on a fresh search page. It now has two tabs. The latter is the normal 'system:similar to' that takes file hashes, if you need to do some manual lookup between imported files, but the first panel is now essentially a paste button. If you copy image data or a file path to your clipboard and paste it there, it'll calculate the similar files info and embed it into the search predicate, letting you search for anything that looks similar to what you have in your clipboard. Give it a go!

I also added a search cache to the main 'similar files' search system. Repeat searches should be massively faster, and searches you re-run with increased distance, like 0 to 4 to 8, and the background similar files search should also accelerate naturally as the cache populates. My 10k file test client's similar files search sped up about 3-4x! I'm not sure what a million+ file client will do, but let me know what you see.
Open

R: 1 / I: 0 / P: 8

Version 528

https://youtube.com/watch?v=I2lgy4_Quko [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v528/Hydrus.Network.528.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v528/Hydrus.Network.528.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v528/Hydrus.Network.528.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v528/Hydrus.Network.528.-.Linux.-.Executable.tar.gz

NOTICE! Version 527 had special update instructions. If you are updating from 526 or earlier, please consult the post here: https://github.com/hydrusnetwork/hydrus/releases/tag/v527

I had an ok week. A bit short on work time, but there's some simple quality of life and bug fixes. The macOS App is also fixed.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

If you regularly use a custom slideshow time, you can now edit the quick-select times shown in the media viewer menu under options->media. It is hacky comma-separated numbers, like the zoom intervals, but have a fiddle and you'll get it.

I improved a bunch of the file search cancel tech. If you put in a search with 'system:imported since 6 months ago', and you then update it before it finishes, it should cancel that ongoing search and start the new one very quickly. This _should_ also vastly improve Client API browsing latency for these situations, since the cancel tech is supposed to fire when a search request is dropped, but let me know how it works IRL.

The time-based system predicate parsing is also more flexible. You can type 'since' or 'before' rather than having to figure out which '<' or '>' you mean, and putting a colon in or 'ago' on the end should parse more reliably. My ideal is to get the text that a system predicate copies to clipboard perfectly parseable. I'd also, fairly soon, like to plug system predicate parsing into the normal file search autocomplete so more users can play with this.

The macOS App had a couple problems last week, with the client->hydrus_client change, and it wasn't booting correctly. Thanks to the users who helped figure this out. Please just install as normal, you still have no special update instructions for the v527 step (it'll just work now!).

next week

Mostly more simple work like this. I'm toying with the idea of a 'paste file/bitmap onto the client, it then does a similar-files search without importing the file' process, basically a 'sauce?' command from your clipboard, which might actually be real easy to add, but we'll see.
Open

R: 1 / I: 0 / P: 8

Version 527

https://www.youtube.com/shorts/xK5SJ6CuezY
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v527/Hydrus.Network.527.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v527/Hydrus.Network.527.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v527/Hydrus.Network.527.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v527/Hydrus.Network.527.-.Linux.-.Executable.tar.gz

I had a good week working on important updates to the build and a new media player prototype.

This build has special update instructions! All users who update from 526 or earlier to 527 or later will have to do this once!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

build update

A couple of weeks ago, I tried updating OpenCV (our image library) for everyone, but it caused a dll conflict and the program wouldn't boot. I am trying again this week, and since we have to do this change, I am also rolling in a long-planned change to the main exectuable names. 'client' is now 'hydrus_client' and 'server' is now 'hydrus_server'. Some users will have to do a 'clean install', and most will have to update their shortcuts or launch scripts.

As always--but particularly this week--please make a backup before you update. The following instructions are simple, but if anything goes wrong, you can always roll back to a safe backup.

Ok, the instructions, by situation, are:
- If you use the Windows installer, update as normal. It will do the clean install for you. Your start menu 'hydrus client' shortcut should also be overwritten with one to the new executable, but if you use a custom shortcut, you will need to update that yourself.
- If you use one of the normal extract builds, do a clean install. You also need to update any program shortcuts.
- If you use the macOS app, there are no special instructions. Update as normal.
- If you run from source, git pull as normal. If you haven't already, feel free to run setup_venv again to get the new OpenCV. Update any custom launch scripts to point at the new hydrus_client.py file.

A 'clean install' is as here: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs
You are basically going to be deleting everything except your database (to clear out old dll files) and then updating as normal.

To update your shortcut or launch script, simply edit it, and where it says 'install_dir\client.exe', just rename it to 'install_dir\hydrus_client.exe'. For Linux users, that'll be 'install_dir/client'.

If you have a custom firewall rule for the Client API, you'll need to update it for the new executable name too.

And that's it! Let me know if you have any trouble.

QMediaPlayer

This is for advanced users only. Also only users who run from source and use Qt6!

I integrated Qt's built-in media player this week. I hope that in future it may be a fallback for users who cannot run mpv. Unfortunately, while it worked surprisingly well when it worked, it was also very crashy. I've got it stable when running from source, but it seems to insta-crash when in a build, so that's something to figure out later.

If you are an advanced user who runs Qt6 from source, you will now see the new experimental Qt Media Player as a selectable viewer under options->media for audio, animations, and video. It has a number of limitations (check the changelog for the full list), but it does work in most cases. Have a play with it and let me know how you get on.

next week

This week was a lot of work for what felt like little outcome. I was disappointed that QMediaPlayer crashed in the build. In any case, I'm short on getting small things done, so I'll return to that. Just some little fixes and quality of life.

I am taking my vacation week on the 8-14th of June, so I have four releases left before then. I won't go for anything huge, but try, if I can, to tie up some loose ends.
Open

R: 1 / I: 0 / P: 8

Version 526

https://youtube.com/watch?v=2_fxNRtmyAE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v526/Hydrus.Network.526.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v526/Hydrus.Network.526.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v526/Hydrus.Network.526.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v526/Hydrus.Network.526.-.Linux.-.Executable.tar.gz

I had a mixed week. I was a little short on time, so this is a simple bugfix and quality of life release.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

When you petition in the manage tags/siblings/parents panel on a repository, it typically asks you for a reason for the change. There are some fixed reasons, but now the system will remember the last five custom reasons you enter too. I hope this saves some typing/pasting work for people with clever petitions to make. You can change the number of recent reasons saved (including setting to 0 to disable) under options->tags.

Every multi-column list in the program now has a 'reset the column widths for this type of list' entry in their header right-click menus. It also resizes immediately (an old debug routine that did this for all lists needed a restart). If your thread watcher list goes crazy wide, it should be fixable now! Let me know how it goes.

I added the basic media keys you find on many keyboards to the shortcut system--previous, next, play/pause, volume up, volume down, and mute. On Windows, these keys seem to have a global hook that fires regardless of how I capture it too, so they may not be so useful yet, but feel free to play around with them.

I fixed a stupid logical typo in my new date calculating code that was breaking system time predicates with the same number of months as the current month (like "system:modified time before 2 years 5 months ago" in May). Sorry for the trouble!

next week

There is going to be an important update next week!

Last week, I tried to update OpenCV for everyone, but it caused a dll conflict in the updated build and I quickly rolled it back in a hotfix. We are going to try again next week, which will require 'extracting' users to perform a clean install. Since we are making that change, I have decided to also roll in a long-planned renaming of the program executables. 'client' will become 'hydrus client' and 'server' 'hydrus server'. This part will require updating of custom shortcuts and launch scripts. I will reiterate all this with specific instructions for all platforms in next week's release, and I'll drop reminders for several weeks after.

Next week is also a 'medium size' job week. If I have time and energy, I'd really like to see if I can get the Qt media player working as an mpv fallback.
Open

R: 2 / I: 0 / P: 8

Version 525

https://youtube.com/watch?v=MgNCjYXCxOc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v525a/Hydrus.Network.525a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v525a/Hydrus.Network.525a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v525a/Hydrus.Network.525a.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v525a/Hydrus.Network.525a.-.Linux.-.Executable.tar.gz

There was a problem with the initial v525, so I rolled back the OpenCV update in this new v525a. If you installed v525 and cannot boot, please delete 'install_dir\cv2\cv2.cp39-win_amd64.pyd' or just do a clean install! If you are an advanced Linux user and use the built release above, please check the note at the top of the old v525 release here: https://github.com/hydrusnetwork/hydrus/releases/tag/v525 . Thank you!

I had an ok week. There's a mix of small work and some library updates for everyone.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

new library versions

Some advanced users have been testing new library versions for Qt (user interface), OpenCV (image processing), and, on Windows, mpv (audio/video display). There haven't been any reported problems so far, so I am rolling these new versions into the regular build and automatic setup scripts for source users. If you use one of the builds above, you don't have to do anything special to update--with luck, you'll simply get some subtly improved UI, images, and video. If you are a source user, you might like to run the 'setup_venv' script again this week after pulling, and it'll update you automatically. If you are a Windows source user, you might also like to get the new mpv dll, which requires a simple rename to work, as here: https://hydrusnetwork.github.io/hydrus/running_from_source.html#built_programs

I don't expect this to happen, but if you are on an older OS, there is a chance you will not be able to boot this release. If this is you, make sure you make a backup before updating, and then if you have problems, roll back and let me know, and we'll figure out a solution.

If you can't boot after an update, but a fresh extract on your desktop works fine, then you probably have a dll conflict, so run a clean install: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs

other highlights

I fixed some more time/calendar stuff. Some longer time durations now compensate for leap years and span month-deltas more precisely.

In the media viewer, clicking the 'exif and other embedded metadata' button up top on jpegs now shows their subsampling (e.g. 4:4:4) and if they are progressive.

I fixed an awful bug related to deleting the new inc/dec rating services. If you deleted one of these and found you couldn't load many files, sorry for the trouble--the underlying problem is fixed, and updating will correct any bad records!

The Sankaku downloader gets some new download objects this week to handle their recent shift to md5-based Post URLs. Unfortunately, this is going to cause any sank subscriptions to go slightly bananas on their next check, hitting their file limits since they think they are seeing new URLs. There's no nice solution to this, so you will pretty much just want to wait them out.

next week

More small stuff like this I think. Maybe I'll squeeze some API work in.
Open

R: 1 / I: 0 / P: 8

Version 524

https://youtube.com/watch?v=UdiwBiw5dyo [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v524/Hydrus.Network.524.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v524/Hydrus.Network.524.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v524/Hydrus.Network.524.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v524/Hydrus.Network.524.-.Linux.-.Executable.tar.gz

I had a good week working on bug fixes, quality of life, and more timestamps.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

timestamps

I've added timestamps to the sidecars system. You can import/export any timestamp the file has, so it is now possible to migrate various times (particularly archive and imported) from one client to another. I still need to add some more things to sidecars, e.g. inbox status and ratings, but we are almost at the point where we can do complete client-to-client transfers of all the major metadata. I have plans to make this ultimate goal simple with templated one-click jobs and no need to repeat the file import--just a raw JSON file metadata import/export.

Also, in the duplicate system, file modified dates can now be merged, and by default they will--for 'this file is better', then an earlier timestamp on the worse file will be applied to the better file, and for 'they are the same', both files will get the earlier of the two. If you don't want this to happen, check your duplicate merge options and set 'make no changes'.

Also in duplicates, if you have URLs set to merge, then any associated domain timestamps with those URLs will be earliest-merged too.

Lastly, the program can broadly support pre-1970 timestamps now. 1970-01-01 is a special date in computing, and supporting times before it can be tricky, but I've fudged some general support. It is a bit of a meme, but maybe it'll be neat to apply '1503' to old paintings. The code I was working with says it is good back to 1 AD, but it seems like it can actually go further, ha ha. Anyway, have fun, and let me know if it fails. I know that the '1,504 years 2 months ago' strings are a little inaccurate--it turns out I haven't been counting leap-year days in those calculations, and it shows on the longer durations.

other highlights

Export Folders now have popups while they work. You can turn them off for regular runs, but now Import and Export Folders force-show their working popups when you manually start them from the file menu. These popups have stop buttons, so you can cancel Export Folders mid-job now too. Also, I removed the stupid and confusing 'paused' vs 'run regularly?' duality. Export Folders now just have 'run regularly?'.

I added some special predicate labels, 'system:ratio is square/portrait/landscape', for =/taller/wider than 1:1. They are in the quick-select list for system:dimensions too.

next week

I pushed it too hard this week and exhausted myself. Thankfully, next week is due to be code cleanup, so I'll try and take it easy and just dive into some simple refactoring work and do some small jobs here and there.
Open

R: 1 / I: 0 / P: 9

Version 523

https://youtube.com/watch?v=LGs_vGt0MY8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v523/Hydrus.Network.523.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v523/Hydrus.Network.523.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v523/Hydrus.Network.523.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v523/Hydrus.Network.523.-.Linux.-.Executable.tar.gz

I had a great couple of weeks working on file timestamps.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

timestamps

Unfortunately, although I did a ton of work here, not much of it is interesting or even visible! Essentially, two weeks ago, the different timestamps your files have--stuff like import time and archived time--were stored all over the place, and now they are stored together. When they are consulted, it now all happens down the same pathway, and I have written a proper update pipeline using this new system.

The user upshot is that you can now right-click->manage->times on a file and edit all its timestamps. This means archived, imported, deleted, previously imported (for undelete), file modified, domain modified, and last viewed times. This new dialog is basic and only for single files for now, but we can now see these things any time and edit little problems. If you end up using this dialog a lot, there's a media shortcut action, 'manage file times' for it.

I've also added 'system:archived' for the 'system:time' stub. It works just like the other time predicates.

I had hoped to integrate this new timestamp manager and update pipeline into the sidecar and Client API systems, but there was so much cleanup to do that I just ran out of time. I'm very glad I did this work, and there's now a much nicer base to work with and extend time tracking in future--like if I want to figure out a 'fill in some good guesses for retroactive archived times', it wouldn't be so much of a hack now--but there is more to do.

other highlights

All multi-column lists across the program now show ▲ or ▼ on the column header they are currently sorted by. This is something I meant to add for ages; now it is done.

All menus across the program now send their commands' longer description text to the main window's status bar. They also show it on tooltips now. These descriptions have been in the program the whole time, but many were difficult or impossible to see. Let's see how annoying the tooltips are--I'll add an option or turn them off if they keep getting in the way.

I fixed a stupid bug from v521 that was causing the advanced deletion dialog to always set the default reason. Sorry for the trouble--it was a testing issue!

next week

A mix of small work, just general catchup. I'd like to also get timestamps working in sidecars and/or the Client API, but we'll see.

R: 207 / I: 17 / P: 9

Q&A Thread

Please feel free to ask questions about hydrus here.

As a reminder, the help and getting started guide is here:

https://hydrusnetwork.github.io/hydrus/help/index.html
Open

R: 2 / I: 0 / P: 9

Version 522

https://youtube.com/watch?v=2AjrLci_AL8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v522/Hydrus.Network.522.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v522/Hydrus.Network.522.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v522/Hydrus.Network.522.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v522/Hydrus.Network.522.-.Linux.-.Executable.tar.gz

I had a good week mostly working on technical things for advanced users.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

notes in sidecars

You can now import and export file notes with sidecars! There's a technical compromise I had to make here, which is that a note in a sidecar comes in the form 'name: text', both for import and export. If you want to import a whole bunch of notes, you'll need to wangle them into this format, or, if you can, use string processors to get them into that format. There's an escape sequence, ':\ ', for if your name has a colon in, too. Have a play with it and you'll see how it works.

The changelog and a new section in the sidecars help talks about it more, including issues about multi-paragraph notes. I suspect I'll be revisiting this, so let me know how it works for you!

neat user submission

This is advanced.

A user contributed two cool new things for the parsing system:

First, there's a new content parser type, 'http headers', which lets you parse something to be submitted in all subsequent URLs created by the current job. It should let us figure out some unusual tokens and pseudo-login issues.

Second, there's a new String Conversion type that lets you calculate the hex hash of any text for the normal hydrus suite of hashes--md5, sha1, sha256, sha512.

more http header stuff

This is advanced.

On my end, I added full custom http header management to the Client API this week. Basically anything global/domain related that you see under network->data->review http headers can be done via API now. This has been a long time coming, and I am glad it is finally out the door. It should allow for setting up some complex site access via the API.

next week

Next week is a 'medium size job' week. I'm not sure what I want to do. I've been thinking of a complete pass to let us edit timestamps, but there's a million other things in my list, so I'll think about it. I'm still a little busy with some IRL stuff, so it might end up being a two-week release.
Open

R: 1 / I: 0 / P: 9

Version 521

https://youtube.com/watch?v=F7gXLqMe5Bw [Embed]
Sorry these links were for v520 by accident earlier!
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v521/Hydrus.Network.521.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v521/Hydrus.Network.521.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v521/Hydrus.Network.521.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v521/Hydrus.Network.521.-.Linux.-.Executable.tar.gz

I had a good week mostly cleaning things and fixing some unusual issues.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Building on last week's custom 'sibling' connection, options->tag presentation now lets you edit the 'OR' connector too, and you can now set a specific colour for them both using the existing namespace colours.

If you don't like file viewing statistics tracking your viewing in the archive/delete filter, you can now turn it off. Also, when you are looking at media with duration, the 'max' time it can record for that video or audio is now the larger of your 'max' setting and five times' that duration. So, if you are capped to ten minutes max normally, but you loop or skip around on a twenty minute vid for thirty mins, it'll save the full thirty mins. I'd like to add some pause tech in future too, to account for minimising and pausing media with duration. Thanks to the user who provided feedback on this stuff.

The file delete dialog should be better, in simple and advanced mode, at not overwriting existing delete reasons, which happens most when you manually delete from the trash. If you use this stuff a lot, let me know how it goes.

The rest is a bit advanced and niche. Hydrus parsers can now support unnamespaced tags with colons in, essentially an extension of the existing ':p' tags, and users can enter these tags too by starting a tag with a colon. If you make parsers, check out the changelog for how tag content parsers can now support 'any' vs 'unamespaced' namespace. I also fixed some weird file domain handling in importers.

next week

I really didn't have time for everything I wanted this week, so I'd like to figure out custom/remembered 'reasons' for the petition workflows, more file viewing stuff, http header editing in the API, and maybe notes support in the sidecar system.
Open

R: 2 / I: 0 / P: 9

Version 520

https://youtube.com/watch?v=outcGtbnMuQ [Embed]
If you are an engineer but didn't see the above vid yesterday, I strongly recommend it, the whole thing. We're clearly in the inflection point of this tech right now, and if you aren't experimenting with integrating some variant of it in some way into your work workflow, I think it is time. I just started using it for some scripting jobs I had been putting off this week and it honestly really helped out. There's problems aplenty in the whole space, but I'm pretty AI-pilled right now.
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v520/Hydrus.Network.520.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v520/Hydrus.Network.520.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v520/Hydrus.Network.520.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v520/Hydrus.Network.520.-.Linux.-.Executable.tar.gz

I had a good week. There's a mix of bug fixes and improvements to quality of life, mostly in the tag autocomplete interface.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

So autocomplete has a couple new things. First off, there's a new 'all files ever imported/deleted' entry under the file domain button. This searches everything your client has ever seen, runs super fast, and works with 'all known tags'. Also, when editing tags, I've tightened up the selection rules and reduced the sibling spam--now, you should get the 'ideal' of what you typed at the top, then what you actually typed (if that differs), and then normal results. Also, the various paste buttons tucked into edit autocompletes now only ever add items--if any of what you paste is already in the accompanying list, it won't be removed now.

And I changed the '(displays as xxx)' sibling indicator. It is now a simple unicode arrow, and because I wanted to play with gradients, I made it fade its colour from one namespace to another. If you don't like the text or the fade, you can change them under options->tag presentation.

There's a new Deviant Art downloader, thanks to a user's work, that pulls the 'original' quality from the 'download' button, which some creators turn on for logged-in users. There's also five new File URL URL Classes, which you can search with system:known url, that will categorise your existing DA downloads according to their quality. Enterprising advanced users might like to play around with this!

next week

Next week is slated for code cleanup. I want to do some boring database breakup work!
Open

R: 1 / I: 0 / P: 9

Version 519

https://youtube.com/watch?v=NNNRlPfoC7I [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v519/Hydrus.Network.519.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v519/Hydrus.Network.519.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v519/Hydrus.Network.519.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v519/Hydrus.Network.519.-.Linux.-.Executable.tar.gz

I had a good couple of weeks prototyping a new rating service.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

inc/dec ratings

So, under services->manage services, there's a new 'inc/dec rating' (for increment/decrement). It works pretty much like all the other ratings, and goes in the same places, but it is a simple number counter--left-click to add, right to subtract. Middle-click lets you edit directly. I've wired it up for system:rating, too, with only one oddness--since every file starts at count 0, there's no concept of 'not rated' for this service.

If you want to count how often an image makes you smile or anything else, give it a go! I mostly did this for fun, but let me know how it works out.

other highlights

The manage siblings and parents dialogs now both try to show more of the full 'chain' of related pairs as you enter new tags. The manage parents one also has a new checkbox that let's you decide if you want to really show everything, including cousins (which can be overwhelming on the bigger tags), or just (grand)children and (grand)parents. I'm planning an overhaul to these dialogs, one where they don't have to load everything on boot, and this is a first step.

I fixed my native animation renderer drawing things too small when your UI scale is >100%!

Server janitors get a nicer account modification dialog this week, and a return of the old 'superban' functionality, which deletes everything an account has uploaded. Check the changelog for full details!

next week

I'm sorry to say I'm exhausted and lagging a bit. I'll clear some small work, nothing too special!
Open

R: 3 / I: 1 / P: 9

Version 518

https://youtube.com/watch?v=lDWfgUjlEMk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v518/Hydrus.Network.518.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v518/Hydrus.Network.518.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v518/Hydrus.Network.518.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v518/Hydrus.Network.518.-.Linux.-.Executable.tar.gz

I had an ok week. Tag autocomplete search should be a bit nicer to work with today!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

tag autocomplete

I rewrote the part of tag autocomplete search that handles showing you early results. The whole thing works better and is smoother.

First off, every time you type something like 'cat' and the search is taking a while, you should now always get 'cat' and 'species:cat' and anything else that might match, with counts, and your selection will be preserved when the full results come in. Quick-entering a tag on both search pages and the manage tags dialog should be faster, easier, and more reliable. This is roughly how things used to work, a long time ago, but the various updates and new options I've added over the years made it patchy and buggy--it should be working again, and with some new quality of life improvements.

Secondly, I've improved sibling lookup in the manage tags dialog. You should now always see siblings, on the pre-results above and the final results, even if the siblings have no count. I think we might want some options for handling how these display and rank (e.g. 'always put the ideal of what I typed at the top of the list'), so see what you think.

Lastly, when you search for tags on a page of thumbnail results, you now get some pre-fetch results very quickly! The full results, which have extra sibling lookup data, will fill in a few seconds later, but, fingers crossed, there will be no more huge waits for anything on thumbnail tag lookups!

misc

The system:time predicates now support hh:mm, like '10:30pm', on the calendar search! Should be easy to cut a day in half now.

I believe I have optimised some content update code. If you have a client with a large page of thumbnails, particularly if that page uses collect-by, let me know if your subscriptions and/or 'upload pending mappings' run a bit smoother.

If you are an advanced Windows user, please check out the bottom of the changelog for an mpv test. I'd like to test out a new dll on a wide range of machines.

next week

My ongoing family IRL stuff is winding down. I'm still going to have extra responsibilities for another month or so, but it shouldn't be too bad. I'd like to have a medium-size job week, and I want to have some fun, so I'm going to try to make a 'click to increment' integer rating service.
Open

R: 1 / I: 0 / P: 9

Version 517

https://youtube.com/watch?v=q4veLQfXFLI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v517/Hydrus.Network.517.-.Linux.-.Executable.tar.gz

I had another short week, but I'm generally happy with the work, which was a mix of different things.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

Thanks to a user's work, Export Folders now support exporting to symlinks! If you are on Windows, you may need to run hydrus in Admin mode for any symlink exporting to work (or, on Windows Pro, I believe you can mess around with Group Policy Editor to give normal user accounts the permission).

The newly upgraded 'Related Tags' shouldn't suggest superfluous siblings or parents any more! I'm pretty sure I got the logic right here, but let me know if it is still suggesting the wrong things somewhere.

Hydrus can now show a checkerboard pattern behind a transparent image instead of the normal background colour. There's two new checkboxes, in options->media and ->duplicates, to control it--by default I have set it just to show in the duplicate filter, where it is useful to have transparency pop when you are comparing two similar files.

next week

Yet again, I'm a little busy with IRL again next week, but otherwise should be normal. I'll do more work like this! Thanks for your patience!
Open

R: 1 / I: 0 / P: 9

Version 516

https://youtube.com/watch?v=d1zDNBIswek [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v516/Hydrus.Network.516.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v516/Hydrus.Network.516.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v516/Hydrus.Network.516.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v516/Hydrus.Network.516.-.Linux.-.Executable.tar.gz

I had a good week that was unfortunately short again. There's some options to tune the new 'related tags' search and some misc little work.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

related tags

I have added three new settings to options->tag suggestions for related tags. They are fairly advanced and deal with the technical details of the search, so I only recommend poking around if you have been enjoying this system and want to change how it works.

First off is a 'concurrence threshold'. This is a percentage throttle that determines how 'related' a suggestion tag needs to be to its search tag to be included. The higher you make it, the fewer--but more relevant--suggestion tags you will get.

Next are two controls that boost or reduce the power of different namespaces in the search. If you want the character tags already on the file to do most of the searching, or if you want to see creator tags most of all, you can alter the weights here. You can also easily set a zero weight to say 'I don't want to see any filename: tags suggested' and so on. I have set them up with some reasonable defaults, so tweak what I've put in and see what you like.

I also fixed a stupid bug where you couldn't hide related tags while in side-by-side mode. Sorry for the trouble!

misc

APNGs that play only once now do that!

The new sidecar system now has import/export/duplicate buttons, so you can save and transfer your work! A favourites system will come in the future too, I'm sure.

The Client API has some more 'king' info in the 'get_file_relationships' command.

next week

The big IRL thing that I had to do is now behind me. I've got about six more weeks of lighter 'hydev has to do some family responsibility stuff', and then I am back to normal schedule. Next week I would like to continue on the small jobs, just to catch up on things that I have recently missed.
Open

R: 0 / I: 0 / P: 9

Get Professional Voiceovers in 3 Clicks with Speechelo

Tired of spending hours searching for voiceover artists? Save time and money with Speechelo. Instantly transform any text into a human-sounding voiceover with just 3 clicks.

Speechelo uses AI to generate professional-quality voiceovers in various languages and accents. Choose from male or female voices, and get your voiceovers done in minutes, not hours or days.

Transform your text into a professional voiceover now with Speechelo.

https://3e9bfyidnydw3k8l34n2v1snci.hop.clickbank.net

Best regards
Thor Andersson
Open

R: 1 / I: 0 / P: 10

Version 515

https://youtube.com/watch?v=Q4brYfcHQR0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v515/Hydrus.Network.515.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v515/Hydrus.Network.515.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v515/Hydrus.Network.515.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v515/Hydrus.Network.515.-.Linux.-.Executable.tar.gz

I had a challenging week with little work time, but I polished last week's related tags work.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

related tags

I've improved the 'related tags' test I did last week and brushed up the UI, and I'm happy enough with it that I am making it default-on for all users. You will see it after you update in your manage tags dialogs. If you do not like it, you can turn it off or customise its timings under options->tag suggestions!

For the technical details, I've cleaned up the new algorithm a bit, made it run faster in all cases and massively faster for very large-count tags, and now it works on the same 'cancel after 1,200ms' tech the old buttons ran on. Those buttons now use the new algorithm exclusively, there's a new label that reports how particular searches went. There's also a 'my files' vs 'all known files' on/off button to change the search domain ('new 1' vs 'new 2' in the test last week).

Also, if your suggested tags are set to show 'side-by-side', they now get nicer boxes around them, and if they are set to show in a notebook, you can set which is the default page under options->tag suggestions. Let me know how you get on with it!

next week

Unfortunately some stressful IRL landed on my head this week, and I'll have some extra responsibilities for the next 6-8 weeks. Nothing dreadful, but it will cut into my work time a bit. Please expect some more light and possibly skipped releases for the next bit, thanks!

Next week I'll see if I can just do some simple code cleaning and more small jobs.
Open

R: 4 / I: 0 / P: 10

Version 514

https://youtube.com/watch?v=PjfkAuUUq5w [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.Linux.-.Executable.tar.gz

I had a good couple of weeks doing a variety of work. The changelog is long, mostly fixes and improvements that don't change much, but if you use the twitter downloader or the Client API, make sure to read before you update.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

twitter

Twitter changed their policies regarding third-party interfaces recently, and it seems as part of that they took down the 'syndication' API our nice twitter downloaders were using. None of them work any more!

A user has kindly figured out a patch, which today's update will apply. It will throw up a yes/no dialog asking whether to do this, which almost everyone will want to click 'yes' on.

Unfortunately, this downloader can only get the first 0-20 tweets in the latest 20 tweets and retweets an account makes!! That's it! When you update, please check your twitter subs to make sure they moved to this new downloader correctly, and you might like to tell them to check more frequently.

I expect twitter to continue in this direction. I don't know if nitter or any other mirrors will be a viable alternative. If any of your twitter subs get posted in any normal booru, I strongly recommend you pursue those instead!

Client API

I am retiring some old data structures from the Client API today. If you use hyshare, please make sure you update to 0.11.2 or newer. If you use hyextract, please update to 0.4.1. If you use Hydrus Companion, I think you'll be good with the latest release for everything. The main changes on my end are to do with tag-viewing and tag-editing, so if you do a lot of that, make sure you have a backup and do a test of everything after you update.

If you have custom scripts, check the changelog. Mostly services are going to be referred to with service_keys in future, so I am removing 'service_name' structures and only supporting 'service_name' arguments in a legacy state. Please move to the /get_service(s) calls to figure out the service keys you need to use, as I will slowly retire the last legacy support later this year. Note that all users can now copy their service keys from review services.

As a side thing, I also rewrote how v413's /set_file_relationships call works. The old 'list of lists' parameter should still work, but a user reminded me that an Object with proper names would be a lot easier to extend in future, so I just rewrote it. This call is sufficiently new and advanced-user-only that I'm ok changing so quick, but if you set something up for it already, check the docs!
Open

R: 4 / I: 0 / P: 10

Version 513

https://youtube.com/watch?v=D5T9RByz9i8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v513/Hydrus.Network.513.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v513/Hydrus.Network.513.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v513/Hydrus.Network.513.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v513/Hydrus.Network.513.-.Linux.-.Executable.tar.gz

I had a good week focusing on adding the duplicates system to the Client API. There is nothing else, so today's release is only for advanced users.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

duplicates with Client API

The main duplicate commands are now available. Obviously check the changelog and Client API help out for full explanation, but the general gist is 1) There is a new 'manage file relationships' permission to gain access, and 2) You can fetch and create relationships, but you can't delete them yet. I will add these commands in the future.

https://hydrusnetwork.github.io/hydrus/developer_api.html#managing_file_relationships
https://hydrusnetwork.github.io/hydrus/duplicates.html#duplicates_advanced

There's some new enums to grapple with, and you can be dealing with two searches at once when fetching potential pairs. Please have a read, try things out, and let me know where it goes wrong. I expose the ugly technical side of things, so if it isn't clear what a king is, I may need to brush up the duplicates help etc...

I expect to keep working on this in the near future, including adding more search tech as I do related 'mulitple local file services' expansions to the Client API, and in the more distant future as I expand alternates. While I was in here this week, I realised I could brush up and clarify some of the edge-case logic too, as I wasn't sure myself on every situation. Again, let me know what seems confusing or wrong for you, and we'll iterate.

next week

A mix of small jobs, and it would be nice to add some remove/dissolve commands for this new duplicates Client API.

On a side note, I migrated to a new dev machine this week, upgrading from a Minis Forum X35G to a TH80. I'm pretty sure the old machine was a 2-core CPU while the new is 8, ha ha ha, so my working session feels super smooth now. I recommend these little machines if you need a decent office computer (the old one also drove two 4k@60Hz monitors no problem), or a younger family member's first big boy computer.
Open

R: 1 / I: 0 / P: 10

Version 512

https://youtube.com/watch?v=jkYg2eV3xBc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v512/Hydrus.Network.512.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v512/Hydrus.Network.512.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v512/Hydrus.Network.512.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v512/Hydrus.Network.512.-.Linux.-.Executable.tar.gz

I had a great week back at work. There's a mix of different things, but particularly duplicate search improvements.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

duplicates

You can now set a search in the duplicates processing page for 'one file comes from search A, the other comes from search B'! It is now super easy to, say, find png vs jpeg pixel duplicates.

I also improved the overall accuracy and precision of the duplicate search logic. When I looked closer at this code, I wasn't always getting pixel duplicates or search distance correct when actually fetching results to show you, or I was selecting bad representatives of larger groups in the final step. This was particularly true in 'show some random potential pairs', which was showing too many files, often outside the search scope. There's some other bells and whistles that are subtle but should make the results feel more 'human'--for instance, in 'show some random potential pairs', if you are set to 'at least one file matches the search', then the first of the random group will definitely match the search, and if you are set to 'both files match different searches', then the first file matches the first and the others match the second.

I have made several changes here, sometimes significantly increasing the CPU load. In testing, things generally seemed to be running at ok speed, but let me know if your IRL complicated situation gets laggy. Also let me know if you discover any miscounts.

misc

When you select thumbnails or right-click on them, the labels in the menu and status bar are a bit nicer--it can now say 'x images' or 'x mp4s' instead of the flat 'x files' in more places, and any time you have a selection of files with duration, it will say the total duration.

v511's supersampled thumbnails now look good on any UI scale. If you have a multi-UI-scale setup, on multiple monitors or not, you should now set the new setting (under options->thumbnails) to your maximum UI scale and hydrus should look good everywhere!

If you want to import or export a .txt sidecar with a separator character other than newline (e.g. comma, for CSV format), you can now do this! You can set any separator characters you like.

The Client API can now refresh pages.

next week

Next week will be a medium-sized job week. I want to spend extra time adding the duplicates system to the Client API.
Open

R: 4 / I: 0 / P: 10

Version 511

https://youtube.com/watch?v=_3ZC45Q82pg [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v511/Hydrus.Network.511.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v511/Hydrus.Network.511.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v511/Hydrus.Network.511.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v511/Hydrus.Network.511.-.Linux.-.Executable.tar.gz

𝕸𝖊𝖗𝖗𝖞 𝕮𝖍𝖗𝖎𝖘𝖙𝖒𝖆𝖘!

I had a good week fixing some things and adding some new options.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

First off, if you uploaded to v510 and had problems uploading to the PTR, sorry for the trouble! I missed something last week with an object update, and it impacted the network protocol. Thanks for the reports--I figured out a serverside patch and we updated the PTR.

I also fixed some logic in 'file import options' where it uses URLs and parsed hashes to predict if a file is 'already in db' or 'previously deleted' before a download. Some edge cases, particularly where many urls were mapped to one file, were not working quite right, and hashes tended to be overly dispositive. Ultimately, this thing should now be better at saving you bandwidth. The options here, which can be seen by advanced users, have updates too--check the changelog and tooltips for explanations.

If you have a large monitor and UI scale >100%, your thumbnails have looked bad for a while. Please check the new 'Thumbnail UI scale supersampling %' option under options->thumbnails. Set it to your monitor's UI scale, and your thumbnails should regen to be crisp. Let me know how it goes!

If you are into clever tag searching, check out the new 'unnamespaced input gives (any namespace) wildcard results' option under services->manage tag display and search. This is a first attempt to bring back the old 'unnamespaced search tags search namespaced variants' tech we used to have. It makes it so typing in an unnamespaced tag will give you the '*:subtag' variants quickly. These wildcards are labelled specially now, too.

next week

I am taking my Christmas vacation week! I'm going to spend some time with family and get lost in vidya. The week after, I'll see about sinking my teeth back into the serverside tag filter. v512 should be out January the 4th.

Thanks everyone!
Open

R: 4 / I: 0 / P: 10

Version 508

https://youtube.com/watch?v=81tEdqBvHVs [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v508/Hydrus.Network.508.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v508/Hydrus.Network.508.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v508/Hydrus.Network.508.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v508/Hydrus.Network.508.-.Linux.-.Executable.tar.gz

I had a great week fixing some bugs, adding hash lookup to the Client API, and finally attaching a Tag Filter to the PTR.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

tag filter

For normal users: the PTR will start filtering out some bad tags automatically, you don't have to do anything.

The big job I wanted to do this week, adding a Tag Filter to the PTR (and other tag repositories), went a lot better than I expected. I split the work into four pieces and got the first one done. If you are a tag repository administrator, you can now edit a Tag Filter for your repo under the services->administrate services->tag repo->edit tag filter menu. Your tag repo will silently discard any tag mapping uploads that do not pass the filter. So, if you don't want 'tagme' or 'source:some-borked-url' tags, you can now filter them at the point of upload and no one has to create or process any petitions in future.

Regular users will sync to the set filter and can see it under services->review services. Users in advanced mode will also get popups on specific rule changes. The next step here will be to expose this filter more, showing it in manage tags and similar UI, not even trying to pend tags that don't match the filter, and retroactively fixing mappings that already exist on the server (including hard-replacing since-sibling'd tags). I'd like to get at least some of this done before the end of the year, but I'm sure some will spill into 2023.

other highlights

There is a new 'file relationships: show x' action in the 'media' shortcut set, which will show a file's potential dupes, actual dupes, alternates, or false positives in a new page, just like the 'show x' actions that are buried in the thumbnail menu. This works in the media viewer too.

I fixed a recent stupid bug in the duplicate filter that meant if you did a batch of work with only file deletes, they were not going through. Sorry for the trouble! The thing now calculates and reports duplicate decisions and manual file deletes separately. I also added a (+50%, -33%) label to the file size comparison statement, just as a test. The numbers are correct, but I'm not sure I like it--let me know what you think.

If you work with the downloader system, URL Classes get two new checkboxes this week, 'do not allow any extra path components/parameters', which stop matching URLs that have too-long paths or parameters. It should help fix up some situations where hydrus was having trouble figuring out which URL Class to match sets of nested URLs to, particularly when the Gallery URL is 'longer' than the Post URL.

With the new 
/get_files/file_hashes
call, the Client API can now do lookups between sha256, md5, sha1, and sha512 hashes. If you provide some hashes in one of these types, the API can now give you known matches for any of the other types. Let's say you have a bunch of sha1 hashes and want to see if you have them imported, or you have a bunch of sha256 hashes for files you have and want to get their known md5s for an external site lookup, this is now possible. The client typically knows all four hashes for every file it has ever imported, so you can do your own 'already deleted' checks too.

next week

More like this. Some small work and some more janitor/serverside stuff. Only three more releases this year, so I'll try to keep things simple.
Open

R: 4 / I: 0 / P: 10

Version 510

https://youtube.com/watch?v=8H40p44oo1U [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v510/Hydrus.Network.510.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v510/Hydrus.Network.510.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v510/Hydrus.Network.510.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v510/Hydrus.Network.510.-.Linux.-.Executable.tar.gz

I had a good week improving notes and UI quality of life. It is Hydrus's 11th birthday today!

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

File notes got several updates this week, mostly for copying notes from one file to another. First off, the 'duplicate metadata merge options' finally support notes! All users will get some basic 'yes please, merge my notes' defaults set, copying from worst to best and copying both ways when the files are the same quality. If you do a lot of duplicate filtering and have been waiting for this, double check your options (under the duplicates processing page) to make sure you are happy. Relatedly, the notes dialog now has copy/paste buttons for copying from one file to another. To get this all to work, these actions use some Note Import Options tech to handle conflicts, generally by extending existing sub-notes where possible and renaming on conflicts. If you want to work with notes in the Client API, this 'easy merge' tech is also optionally available in the Client API. Let me know how this works for you--I think I am going to have to adjust the logic to handle two-way related note merges a little better, so feedback on IRL situations would be great.

I significantly improved how the taglist handles shift-selections and drag-selections. A bunch of position and logic jank is cleaned up, and now you can 'undo' a multi-selection, just like I did for thumbnails recently. Best way to test this is just click and drag down the list, and then move your mouse up--it undoes the selection as you move up, and it remembers what was originally selected before! Also, holding ctrl down to start a deselection drag is cleaner and more human. Give it a go, and you'll feel the difference.

next week

Just one more week of work and then I am on my vacation. I'll do some more cleanup/small jobs and try to catch up on github bug reports, which I have been neglecting.
Open

R: 6 / I: 0 / P: 10

Version 509

https://youtube.com/watch?v=pLlAvGG6AYg [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v509/Hydrus.Network.509.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v509/Hydrus.Network.509.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v509/Hydrus.Network.509.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v509/Hydrus.Network.509.-.Linux.-.Executable.tar.gz

I had a good week mostly fixing bugs.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

important fixes

When adding the tag filter last week, some users got errors appearing on shutdown. Thankfully, the problem was mostly harmless, but it was a surprise, and annoying, and I regret it. It believe in a legacy issue that some users had a different internal datatype that made it difficult to save the synced tag filter. I am confident I have fixed it this week, but let me know if you have any more trouble.

The macOS app has had trouble booting since v505. Thanks to help from several users, we finally pinned down the issue and got a fix. Also, we are switching the macOS release to a slightly different version of Qt, the UI library, and the .dmg is now half the size (548MB->258MB!).

other highlights

There are several menu-spawning buttons across the program that can be changed by scrolling the mouse over them. Good example is the 'ascending/descending' buttons for file sort or the taglist sort. If you find this behaviour annoying, you can now turn it off under options->gui!

If you run from source and are willing to test new things out, you might like to run setup-venv again this week. The advanced install route now has a '(t)est' Qt version that tries out 6.4.0. This new release has a heap of bug fixes, but it caused problems when it first launched, and I'd like to get some feedback on it before again trying to roll it into the built releases.

I fixed a bunch of bad thumbnail handling/rendering, but there is more work to do.

next week

A cleanup week. More small fixes and UI tweaks, nothing too dramatic.
Open

R: 4 / I: 0 / P: 10

Version 507

https://youtube.com/watch?v=21uh28Z77Xg [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v507/Hydrus.Network.507.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v507/Hydrus.Network.507.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v507/Hydrus.Network.507.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v507/Hydrus.Network.507.-.Linux.-.Executable.tar.gz

I had an ok week. There are some bug fixes, more Client API data, and a new tool for server janitors.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

This week is a little slim on updates for normal users. Mostly bug fixes, and updates to the 'running from source' setup scripts, and there is a neat fix/improvement when you apply set-duplicate actions to many thumbnails at once. Check the changelog if you want to read more!

For advanced users, the Client API gets some more of the service name, type, and type_pretty variables, and 'all known tags' appears on the new 'tags' structure in the 'file_metadata' call (I forgot to add it last week!).

For hydrus repository admins and janitors, there is an update to the petition workflow. The petitions page now has an 'account id' field where you can paste an account id and see petition counts (and petitions) just from that account. Servers have to update for this to work.

next week

It felt good to clean some ancient server code this week. There's plenty more I want to get done before the end of the year, so I'll keep focusing on it. This is scheduled to be a 'medium size job' week, so if I can figure it out, I'd really like to have a basic tag filter implemented so the server can auto-discard tags on upload. We'll see!
Open

R: 1 / I: 0 / P: 10

Version 506

https://youtube.com/watch?v=YJu0iXn-T_U [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v506/Hydrus.Network.506.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v506/Hydrus.Network.506.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v506/Hydrus.Network.506.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v506/Hydrus.Network.506.-.Linux.-.Executable.tar.gz

I had a great week catching up on a variety of small cleanup, fixes, client api, sidecars, and ui quality of life.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

highlights

system:rating gets the same overhaul system:time had recently. Some nicer labels and clearer interaction. If you use this a lot, have a poke around, because the click logic has changed a little--to say 'is not rated', you now say the rating 'is' 'nothing'.

The media viewer has some new shortcut actions for 'zoom: 100/canvas/default'.

If you right-click on a file that has more than one recorded modified date, you can now see them all on the top submenu. This should be true for anything downloaded inside the client in the past few months. Notice how that the typical modified date you see is the most reasonable minimum value of all recorded.

The new expanded sidecar import/export system gets more options for the sidecar filenames. You can now take out the '.jpg' component, or, if you need to, go crazy with a full String Converter on the filename. There's also some test UI so you can enter an example media filename and see the expected sidecar filename.

The Client API's '/get_files/file_metadata' gets several updates this week. If you do API stuff, please check it out. There's a new combined 'tags' structure, and I am retiring the old 'service_blah_to_blah_tags' structures over the next eight weeks, so watch out! EDIT: I did one part of this too quick. If you use hydrus web, hyextract, or hyshare, or hydrus companion, wait a couple of days before you update to 506--the guys who work on these are updating them to catch the change I made here.

The 'running from source' help and scripts get some more fixes and updates for unusual situations. Thank you everyone for the reports and recommendations. If you had trouble with this stuff previously, please try again!

next week

The end of the year is coming fast, and the one thing I wanted to do--janitor workflow improvements--isn't anywhere near where I want it. I am going to try to focus on serverside stuff until I feel better about this, so normal user features may be a little barebones for a bit.
Open

R: 5 / I: 1 / P: 11

Version 465

https://youtube.com/watch?v=QgNEYyOlST0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v465/Hydrus.Network.465.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v465/Hydrus.Network.465.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v465/Hydrus.Network.465.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v465/Hydrus.Network.465.-.Linux.-.Executable.tar.gz

I had a good week. I focused on background improvements to the database. If you have a big client, it will take several minutes to update the database. My 2.4 million file PTR-syncing IRL client took 8 minutes.

misc

First off, some quick fixes: I fixed unnamespaced wildcard tag searches (e.g. 'sam*s'), which were recently not producing namespaced results. I also improved handling in the new ICC profile system when the ICC profile with an image being loaded was completely borked.

Also, it seems the login-required 'gelbooru favourites by user id' downloader recently(?) broke severely--as well as pulling favourite links, it was also parsing and visiting the 'remove from favourites' link and deleting them! I fixed the gelbooru gallery parser to never pull a delete link again, but if you used this parser to grab all your favourites, please check your favourites list, and if you can, dig that downloader back out of hydrus and do a mass 'open sources' on the file log or the thumbnails so you can re-favourite any files that were dropped. Thanks to the users who noticed what was going on here and figured out what needed to be fixed.

I added some new options. The 'default tag service in manage tags' choice is reset this week, and it now starts off working different: it now remembers the last used service the next time you open a tag service dialog. Let's see if this works out, but if you don't like it, you can go back to a fixed default under options->tags. There's also a new checkbox in options->search that lets you default new file search pages to 'searching paused' rather than 'searching immediately'.

ipfs and deleted files search

This is mostly database prep for future multiple local file service expansions.

If you have an IPFS service, you can now search it in a normal search page. Just switch 'my files' on the autocomplete dropdown to your IPFS service and you should be able to search it with tag counts and everything. IPFS works a little differently to a normal file service in hydrus, so this will need some more work to get those workflows integrated. Also, while an IPFS service in hydrus only knows about your pins atm, in future I would like hydrus to harvest more info from external sources so this search space could potentially populate with remote files that you could then command the client to download.

In a related but quieter move, I did the same thing here for a new 'deleted files' umbrella domain. It'll take a few minutes to calculate this search cache on update. This will be of use in the near future when I let advanced users start searching deleted files.
Open

R: 2 / I: 0 / P: 11

Version 505

https://youtube.com/watch?v=Z1MV0WbrNwY [Embed]
Windows release got a hotfix! If you got 505a right after the release was posted and everything is a bad darkmode, get the new one!
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v505b/Hydrus.Network.505b.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v505b/Hydrus.Network.505b.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v505a/Hydrus.Network.505a.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v505a/Hydrus.Network.505a.-.Linux.-.Executable.tar.gz

I had a great couple of weeks fixing bugs, exposing EXIF and other embedded metadata better, and making it easier for anyone to run the client from source.

Full changelog (big one this week): https://hydrusnetwork.github.io/hydrus/changelog.html

EXIF

I added tentative EXIF support a little while ago. It wasn't very good--it never knew if a file had EXIF before you checked, so it was inconvenient, and non searchable--but the basic framework was there. This week I make that prototype more useful.

First off, the client doesn't just look at EXIF. It also scans images and animations for miscellaneous 'human-readable embedded metadata'. This is often some technical timing or DPI data, or information about the program that created the file, but, most neatly, for the new AI/ML-drawn images everyone has been playing with, many of the generation engines embed the creation prompt in the header of the output png, and this is now viewable in the client!

Secondly, the client now knows ahead of time which files have this data to show. A new file maintenance job will be scheduled on update for all your existing images and animations to retroactively check for this, and they will fill in in the background over the next few weeks. You can now search for which files have known EXIF or other embedded metadata under a new combined 'system:embedded metadata' predicate, which works like 'system:dimensions' and also bundles the old 'system:has icc profile' predicates.

Also, the 'cog' button in the media viewer's top hover window where you would check for EXIF is replaced by a 'text on window' icon that is only in view if the file has something to show.

Have a play with this and let me know how it goes. The next step here will be to store the actual keys and values of EXIF and other metadata in the database so you can search them specifically. It should be possible to allow some form of 'system:EXIF ISO level>400' or 'system:has "parameters" embedded text value'.

running from source

I have written Linux (.sh) and macOS (.command) versions of the 'running from source' easy-setup scripts. The help is updated too, here:

https://hydrusnetwork.github.io/hydrus/running_from_source.html

I've also updated the setup script and process to be simpler and give you guidance on every decision. If you have had trouble getting the builds to work on your OS, please try running from source from now on. Running from source is the best way to relieve compatibility problems.

I've been working with some users to get the Linux build, as linked above, to have better mpv support. We figured out a solution (basically rolling back some libraries to improve compatibility), so more users should get good mpv off the bat from the build, but the duct tape is really straining here. if you have any trouble with it, or if you are running Ubuntu 22.04 equivalent, I strongly recommend you just move to running from source.
Open

R: 6 / I: 0 / P: 11

Version 504

https://youtube.com/watch?v=W-59sWasI98 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v504/Hydrus.Network.504.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v504/Hydrus.Network.504.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v504/Hydrus.Network.504.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v504/Hydrus.Network.504.-.Linux.-.Executable.tar.gz

I had a great week focusing on improving sidecar import/export. Importing and exporting tags and URLs with files is now easy.

Qt5

As a reminder, I am no longer putting out Qt5 builds. If you are on Windows 7 (and I heard one instance of 8.1), or a similarly old OS, you probably cannot run the hydrus builds any more. You can still update, but you need to run from source from now on. The good news is that this is now much easier to do on Windows--please check the help here: https://hydrusnetwork.github.io/hydrus/running_from_source.html

Windows source users should also get a nice hydrus taskbar icon this week.

sidecars

There's a semi-advanced feature in hydrus where you can tell the client to look for (on file import) or create (on file export) additional files (sidecars) along with your media. Until now, this has been fixed to 'filename.jpg.txt' files that would include tags for the file separated by newlines. This system expands radically this week.

What was static before is now flexible. You can set up multiple workflows for multiple sidecars in one task, grab or send URLs as well as tags, parse or output formatted JSON instead of .txt, and transform or filter the text at every stage.

This is available on manual imports and exports, and import folders and export folders. For manual imports, it becomes a new 'sidecars' page tab on the 'add tags before import' window, and for all the others it is now a button on the UI. If you are interested in this tech, please check it out and let me know what is confusing. It is quite powerful now, and users who are familiar with the concepts involved (and the parsing system) should find it obvious, but I think I'll need to add some help text and a full help guide for users new to it.

All existing 'neighbouring .txt' settings on import folders, export folders, and the manual file export window should be migrated to the new system when you update. Let me know if there are any problems!

There are many potential next steps here. I'd like a 'favourites' system first, so it is possible to save/load/share workflows, and then we can think about more content types (ratings, notes) and .xml parsing/export.
Open

R: 11 / I: 2 / P: 11

Version 503

https://youtube.com/watch?v=VFyu1qvub7A [Embed]
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Windows.Qt6.-.Extract.only.zip
Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Windows.Qt6.-.Installer.exe
macOS
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Linux.Qt6.-.Executable.tar.gz

I had a good week improving how the program works behind the scenes. The build is more reliable, and it is now easy to run the program from source in Windows.

reliability

tl;dr: The program breaks less now and in the future.

Just as I was hotfixing a fresh boot problem last week, in the intervening time yet another boot problem arrived! We've been hit by several of these in the past couple of months, and I hate causing people this stress, so I decided to put some time into the way the official release is put together. I really hope this will not happen as much in future.

This is mostly technical, but the requirements.txts that prep the program environment are simpler, laid out for easier reading and maintenance, and have frozen version numbers for important libraries. I also put time into the build script itself, doing some cleanup and anti-deprecation work. The issues with PySide6 and dateutil are fixed.

I want the program to be more stable going forward when I click 'publish'. It has been duct-tape-and-a-prayer for too long. Thank you to those who report problems, and please keep them coming if and when this new system fails. I want to keep improving things here, especially too as more users run from source.

running from source

Absent any more issues, I expect this to be the last week I put out a Qt5 build. The Qt6 build is stable with no giant outstanding problems. Users on Windows 7 (and some similarly old Linux and macOS versions) cannot run Qt6, so they will need an alternate solution.

It is now easy to run the program from source in Windows. I have written several batch files to make it simple, no technical knowledge required beyond the ability to open a zip file, and the help has a brand new full walkthrough. If you want to run Qt5, the way to do so is equally simple and clearly labelled. Updating, too, is very similar to how you already update with the Windows extract zip.

https://hydrusnetwork.github.io/hydrus/running_from_source.html

The process is also simpler and clearer for advanced users who want to use 'git', 'venv', and 'pip' manually. If you have been running hydrus from your system python, you might want to check out the help to learn how to set up a venv--it is really easy these days, and you can save yourself a real headache.

I'm obviously interested in reports if any of this fails for you; or if there is anything you find confusing. There is more work to do here, and once we have a workflow people are happy with, I would appreciate if some advanced Linux and macOS users can translate my new .bat files roughly to .command and .sh (I just don't have the familiarity/expertise in these platforms to do this well), so we have easy 'running from source' support for all platforms.
Open

R: 3 / I: 0 / P: 11

Version 502

https://youtube.com/watch?v=SkOAc_eU9b0 [Embed]
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Windows.Qt6.-.Extract.only.zip
Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Windows.Qt6.-.Installer.exe
macOS (updated links, first version had boot trouble)
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v502a/Hydrus.Network.502a.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v502a/Hydrus.Network.502a.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Linux.Qt6.-.Executable.tar.gz

I had a good week fixing bugs and improving UI quality of life.

highlights

First off, the tag autocomplete dropdown has some important behind the scenes changes. The way it is stored and drawn is better, the same updates the popup toaster and media viewer hover windows have had recently. In general, it means a lot less flicker and jank. If you have set autocompletes to embed in normal search pages because of bugs, try letting them float under options->search this week!

Unfortunately, as a consequence of this change, I am disabling autocompletes floating in dialogs. Outside of normal search pages, they now always embed. There are hard technical reasons for this, and design ones too--I never liked how they overlapped apply/cancel--so if you were floating before and liked it, please bear with it for now and let me know if and where it is most inconvenient.

I put some work into the system:time edit predicate dialog. Rather than the confusing '<' and '~=' radiobox, it now says proper english 'since' and 'before' and so on. Some of the calculations behind this are also improved.

'sort files by import time' now works between files that were imported in the same second. If you had a some small-image manga you wanted to sort by import time because it lacked page tags, this now works properly.

I fixed some stupid mistakes that broke a part of the Client API (and Hydrus Web) in v501. Sorry for the trouble!
Open

R: 5 / I: 1 / P: 11

Version 501

https://youtube.com/watch?v=4UeFP0al1IY [Embed]
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Windows.Qt6.-.Extract.only.zip
Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Windows.Qt6.-.Installer.exe
macOS
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Linux.Qt6.-.Executable.tar.gz

I had a good week working on a mix of important bug fixes and UI quality of life.

highlighted fixes

First off, I have rewritten the guts of the popup toaster. It looks the same, but now it is more 'embedded' into the main gui window and I completely overhauled its layout system. I strongly hope that this will eliminate a variety of jank that has occured over the years, from odd activation/focus issues to weird width sizing to unusual popup locations. If you have had trouble with popup messages, let me know how things work today!

Next, due to a change behind the scenes in another library, the Windows build wouldn't boot last week and needed a hotfix. The Linux build seems to have suffered from a similar issue, so I have fixed both. If you had trouble booting v500 Linux, I am sorry--please try again today. The macOS release uses a different system and was unaffected.

I fixed a bug (and sometimes crash) when opening file selection dialogs for users running from source with PyQt6. Thank you for the reports!

I fixed a crash that was affecting many users (including myself a couple times, I now realise) when performing some options-saving events such as pausing/resuming subscriptions, repositories, or import/export folders. I was finally able to reproduce this reliably as I was putting the release together today. Thank you very much to the users for the reports here, I would not have figured it out but for what you discovered. It turns out it was the collect-by control doing it, so its ability to update itself without a client restart is disabled for now.

I fixed a bad bug in the manage ratings dialog, where any numerical ratings that started and ended on dialog ok in the 'mixed' (dark grey colour) state were not leaving them alone but resetting those files' ratings to the minimum allowed value (0 or 1 stars). This was an unusual bug, and I regret it a lot. I think it came in in recent weeks, when I rewrote how some rating states are handled internally.

The Deviant Art downloader broke last week. It seems like they locked away the API we were using. I have reset us to the original html parser, which still seems to work ok. Please check your DA subs and tell them to retry ignored, and let me know if there are any glaring problems.
Open

R: 6 / I: 0 / P: 11

Version 500

https://youtube.com/watch?v=UGZKKXTNcE8 [Embed]
The first Windows build was broken, if you got that please check again--the links have been updated to a hotfix.
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v500a/Hydrus.Network.500a.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v500a/Hydrus.Network.500a.-.Windows.Qt6.-.Extract.only.zip
Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v500a/Hydrus.Network.500a.-.Windows.Qt6.-.Installer.exe
macOS
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v500/Hydrus.Network.500.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v500/Hydrus.Network.500.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v500/Hydrus.Network.500.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v500/Hydrus.Network.500.-.Linux.Qt6.-.Executable.tar.gz

I had a good week-and-a-bit returning to normal hydrus schedule after my personal issues. There are some important bug fixes, particularly for windows crashes, and some neat updates to tag search logic.

Those who use the Windows Qt5 release will want to perform a 'Clean Install' this week: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs

crashes and bugs

I messed up the new mpv version in v499. My golden rule is to never put out bleeding-edge library updates, but without thinking I gave Windows users a dll from late August. This caused instability for a variety of installs, but thanks to some great reports and user testing, we were able to figure out the problem and solution. I regret I wasn't able to roll out an official fix until now, but I will remember this issue for future--never fold in the latest build of anything.

There are two fixes here. Windows Qt6 users simply get a more stable mpv-2.dll today. You don't have to do anything special, just install as normal and your hydrus should be more stable. Windows Qt5 users will be rolling back to mpv-1.dll, so if you are a Qt5 user who updated to v499, you should perform a 'Clean Install', as here: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs . Just follow the guide and you should be good again, but let me know if you have any trouble!

I also fixed a critical issue that was affecting a couple of users with damaged similar file search trees. If you have had 'similar file tree rebalancing' maintenance that seemed to go on forever before locking up your client, this is now fixed. Some related simple errors when the maintenance routine ran into a damaged or looped tree are also fixed.

The Client API now handles disconnects more gracefully. Some logspam is cleared up, and very slow file and tag searches via API now cancel on disconnect just like in the UI (e.g. when you type a new character in autocomplete tag search, it'll cancel the older slower search and start a newer faster one). If you run a busy Hydrus Companion or another Client API application that really hammers your client, let me know how you get on.
Open

R: 18 / I: 0 / P: 11

Version 499

https://youtube.com/watch?v=OIFASfPkw9g [Embed]
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v499/Hydrus.Network.499.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v499/Hydrus.Network.499.-.Windows.Qt6.-.Extract.only.zip
Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v499/Hydrus.Network.499.-.Windows.Qt6.-.Installer.exe
macOS
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v499/Hydrus.Network.499.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v499/Hydrus.Network.499.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v499/Hydrus.Network.499.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v499/Hydrus.Network.499.-.Linux.Qt6.-.Executable.tar.gz

I had a great week catching up on bug reports.

highlights

The changelog is quite long this week. It is mostly smaller bug fixes that aren't interesting to read. Some users have reported crashes in Qt6, and I got one crash IRL myself, which is very rare for me. I am not totally sure what has been causing these, but I fixed some suspects this week, so let me know if things are better.

I am also updating mpv for Windows users this week. It is a big jump, about a year's worth of updates, and I feel like it is a little smoother. It is also more stable and supports weirder files.

And thanks to a user's work, there is an expansion to the twitter downloader. You can now download a twitter user's likes, and from twitter lists, and--if you can find any--twitter collections.

If you are a big 'duplicates' user, there is a subtle change in the potential duplicates search this week. Check the changelog for full details, but I think I've fixed the 'I set "must be pixel dupes" but I saw a pair that wasn't' issue.

full list

- mpv:
- updated the mpv version for Windows. this is more complicated than it sounds and has been fraught with difficulty at times, so I do not try it often, but the situation seems to be much better now. today we are updating about twelve months. I may be imagining it, but things seem a bit smoother. a variety of weird file support should be better--an old transparent apng that I know crashed older mpv no longer causes a crash--and there's some acceleration now for very new CPU chipsets. I've also insisted on precise seeking (rather than keyframe seeking, which some users may have defaulted to). mpv-1.dll is now mpv-2.dll
- I don't have an easy Linux testbed any more, so I would be interested in a Linux 'running from source' user trying out a similar update and letting me know how it goes. try getting the latest libmpv1 and then update python-mpv to 1.0.1 on pip. your 'mpv api version' in _help->about_ should now be 2.0. this new python-mpv seems to have several compatibility improvements, which is what has plagued us before here
- mpv on macOS is still a frustrating question mark, but if this works on Linux, it may open another door. who knows, maybe the new version doesn't crash instantly on load
Open

R: 2 / I: 0 / P: 11

Version 498

https://youtube.com/watch?v=D5T9RByz9i8 [Embed]
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v498/Hydrus.Network.498.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v498/Hydrus.Network.498.-.Windows.Qt6.-.Extract.only.zip
Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v498/Hydrus.Network.498.-.Windows.Qt6.-.Installer.exe
macOS
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v498/Hydrus.Network.498.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v498/Hydrus.Network.498.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v498/Hydrus.Network.498.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v498/Hydrus.Network.498.-.Linux.Qt6.-.Executable.tar.gz

I had a good week. I got started on some long-delayed serverside janitor improvements, and I got so stuck it in was all I worked on! This release has nothing important for regular users, so if you are not involved in running a hydrus server, you can skip it entirely!

server stuff

Only for server admins and janitors!

Improving the janitor workflow is now my 'big job' for the rest of the year. This is the first step of what I'd like to fit in to spare days over the coming weeks and months.

This week I update the repositories so they cache counts of their various metadata. They can now quickly say 'I have 2,124,543 mappings' and so on. If you have a large server, it will take a few minutes to update as it counts everything up for the first time.

Once you are booted, make sure you are in help->advanced mode, and on review services you will have a new 'service info' button. Click it and you can see all the numbers, including the current lengths of the petition queues. Anyone with an account can see these for now. If you want more privacy, I can figure something out, but tbh I think it is probably good for users to be able to see everything here.

The numbers in the petition processing page are fed by this too. No longer will it manually count up and max out at 1,000 petitions--it will deliver the actual number real fast.

Also, a sibling petition can now have both ADD and DELETE rows. This happens if the same account gives the same reason (like 'replacing a->b with a->c') to a sibling petition and a sibling pend. You now see those together, with that shared reason, and action it as one item.

I suspect we'll need some more UI clientside to encourage using the same reason, but for now I have updated manage tag siblings to give the same 'reason' when you replace an existing sibling. Previously, this is where it would give a 'AUTO-CONFLICT...' style reason. Now, those things should be bundled into the same thing you see.

This stuff changes some of the hydrus network protocol. Normally, I would update the network version number, but that requires everyone to update. Since this only affects advanced users, and I expect I'll be doing more in coming weeks, I am not updating the version number. An old client will run into errors if it tries to pull petitions from a new server, but I think a new client will be able to work with an old server. In any case, if you are a server admin or janitor, please update your clients and servers at roughly the same time this week, or you'll get some harmless but annoying parse/404 errors.

As a side thing, as a server admin, if the service info numbers ever get borked, please hit 'regen service info' in your 'administrate services' menu. I've added extensive testing this week to ensure the update routines are mostly good, but I'm sure there are some complicated situations where the counting logic is dodgy. Let me know how you get on with it!
Open

R: 6 / I: 1 / P: 11

Version 497

https://youtube.com/watch?v=0KrpZMNEDOY [Embed]
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v497/Hydrus.Network.497.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v497/Hydrus.Network.497.-.Windows.Qt6.-.Extract.only.zip
Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v497/Hydrus.Network.497.-.Windows.Qt6.-.Installer.exe
macOS
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v497/Hydrus.Network.497.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v497/Hydrus.Network.497.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v497/Hydrus.Network.497.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v497/Hydrus.Network.497.-.Linux.Qt6.-.Executable.tar.gz

I had a good week working on a variety of different things.

The Windows Installer is now Qt6. You do not have to do anything special to update, just do your normal routine. If you are a Windows 7 user, it is very likely you will not be able to run it. You can use the Qt5 extract release for another week or two, but please plan to either stop updating hydrus, move to a newer OS, or consider running hydrus from source.

highlights

I bulked out the 'star' rating shape. The pentagram was a little thin, so I've fattened it back up while still keeping the coordinates good. If you liked the thinner star, you can now set it explicitly as a new shape choice under services->manage services.

Manage tag siblings/parents now has a proper delete button, which should make removing many rows at once easy.

I did some more note parsing work this week, and I updated the Hentai Foundry downloader to grab artist description notes. If you download from HF, you should see new files get notes. I would like to slowly update most of the client default downloaders with note parsing support, so let me know where all this succeeds and fails, and I'll adapt things as we go.

Since we are doing more note work, I also improved the size calculations for the media viewer's note hover window. It still isn't perfect in all cases, but it'll clip the last line of text less often.

New clients now start with ctrl+page up/down as 'move page selection left/right'.

multi-line parsing

This is only for advanced users who make downloaders.

The parsing system now has basic multi-line support. Any formula on a 'notes' content parser or the formulae that do subsidiary page parser 'splitting' will now no longer remove newlines when it grabs text. It makes it possible to parse a multi-paragraph artist comment and have it all end up nicely formatted in the file note you end up with.

I have hardcoded in some additional formatting rules, too. When you select 'string' as the parsing result from a nested tree of html tags, I now insert newlines on 'p' or 'br' tags. I also strip leading and trailing whitespace from every line of a note, and I only allow two consecutive newlines to clip very large paragraph breaks.

The main remaining frustration is the string processing UI has mixed multi-line parsing support in its testing UI. Some of it shows and converts well, but most of it collapses multi-line test content down to a single line.

I have updated the Hentai Foundry parser this week to grab notes. It is ultimately pretty simple, if you want to check it out as an example. I'd like to know what is most frustrating about this. I would like to chip away at the string processing test UI (and any rules that simply do not work well on multi-line content) in future.
Open

R: 3 / I: 0 / P: 12

Version 496

https://youtube.com/watch?v=2jzugX2NMnk [Embed]
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v496/Hydrus.Network.496.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v496/Hydrus.Network.496.-.Windows.Qt6.-.Extract.only.zip
Qt5 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v496/Hydrus.Network.496.-.Windows.Qt5.-.Installer.exe
macOS
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v496/Hydrus.Network.496.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v496/Hydrus.Network.496.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v496/Hydrus.Network.496.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v496/Hydrus.Network.496.-.Linux.Qt6.-.Executable.tar.gz

I had a great week. There's more Qt6 work, and the first version of note parsing is finished.

UI Scaling and Qt6

The media viewer now understands UI scale and will draw images with all available pixels and state the current zoom correctly. Previously, if you were on 150% UI scale, it would draw at the smaller 100% size and then nearest-neighbour scale up. Now it compensates for the UI scale completely, drawing at that size, so if you look at a 4k image on a 4k screen, it'll look the same no matter what your UI scale is.

Some funny math went into this, particularly as I stitched the image tiles together through several different coordinate spaces, so if you are >100% UI scale and see a grey or white line flickering anywhere as you pan an image around, let me know your details.

After last week's nicer thumbnails at >100%, a user let me know that the thumbnail banner text was actually looking pretty bad at 100%. Looking at screenshots of the two situations made it obvious. I chased down what was going on, which turned out to be an annoying technical problem with no easy solutions, and figured out an ok solution. If all your monitors are 100% UI scale, the thumbnail banner text should now be antialiased. It isn't perfect (and even looks blurry with some semi-transparent background colours), so I may revisit this, and it may be the underlying Qt issue causing the ugly font gets fixed one day anyway.

The rating shapes (circles, squares, stars) are now antialiased too, and I re-did the star shape to look better.
Open

R: 4 / I: 0 / P: 12

Version 495

https://youtube.com/watch?v=qBJRJvHYS_k [Embed]
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v495/Hydrus.Network.495.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v495/Hydrus.Network.495.-.Windows.Qt6.-.Extract.only.zip
Qt5 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v495/Hydrus.Network.495.-.Windows.Qt5.-.Installer.exe
macOS
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v495/Hydrus.Network.495.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v495/Hydrus.Network.495.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v495/Hydrus.Network.495.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v495/Hydrus.Network.495.-.Linux.Qt6.-.Executable.tar.gz

I had a great week working on more Qt6 support and getting Note Import Options ready for advanced users to try out.

Qt6

The Qt6 launch last week went generally well. There were a couple of little typo bugs as expected, but most users reported nothing drastic.

I have fixed several issues and also improved graphical quality at >100% UI scale. Qt6 handles UI scale tech much better, but that also exposed all the better where my custom UI was failing. Thumbnails at 125% were looking pretty ugly, with nearest-neighbour scaling, so I knuckled down and did my homework on how all this is supposed to work, and I think I have it fixed. Thumbnails should look ok at any UI scale in Qt6, and their banner text too. My fixes apply to Qt5 too, but as far as I can tell that only really works comprehensively for 100%/200% scale. I will try to tackle the media viewer next week.

If you are an experienced user with a backup, please feel free to try Qt6 out on your real install. If both Qt5 and Qt6 are available, the client will now default to Qt6, so you shouldn't need to do a 'clean install' like last week. My test of this went fine, but if there is some odd dll conflict when you try to boot, check here on how to clear things out and either revert to Qt5-only or try Qt6-only:

https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs

Any reports on failures here would be useful so I can write in any needed 'delete these old files' rules to the Qt6 Win installer.

As a reminder, afaik Windows 7 cannot run Qt6, so don't try it out if that's you. I will switch over to Qt6 exclusively in a few weeks, at which point I'll update the help and talk more about your future options, which will be: stop updating the client; move to a newer OS; run in Win7/Qt5 from source.

Note Import Options

Unfortunately I couldn't fit all this in again, but I've done work I'm happy with and have parts of it ready for advanced users to play with. Fingers crossed, the first simple version of this will be completely integrated next week.

The File Import Options update last week went ok. I messed something up with the Presentation Options, so highlighting a gallery or watcher with a default FIO was always showing the same thing (new or all files) instead of what the default actually said. This is fixed.

I have pushed FIO and TIO further together this week. File Import Options and Tag Import Options are now merged into one button and one dialog across the program. The dialog is tabbed, so you edit both sets of options at the same time, and in future, any new Import Options will work through the same interface.
Open

R: 3 / I: 0 / P: 12

Version 494

https://youtube.com/watch?v=85bvcndvEpo [Embed]
windows
Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v494/Hydrus.Network.494.-.Windows.Qt5.-.Extract.only.zip
Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v494/Hydrus.Network.494.-.Windows.Qt6.-.Extract.only.zip
Qt5 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v494/Hydrus.Network.494.-.Windows.Qt5.-.Installer.exe
macOS
Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v494/Hydrus.Network.494.-.macOS.Qt5.-.App.dmg
Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v494/Hydrus.Network.494.-.macOS.Qt6.-.App.dmg
linux
Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v494/Hydrus.Network.494.-.Linux.Qt5.-.Executable.tar.gz
Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v494/Hydrus.Network.494.-.Linux.Qt6.-.Executable.tar.gz

I had a great week with two big changes. First, there is an optional 'Qt6' version of the program for advanced users to try out, and second, File Import Options has some important updates.

Qt5 and Qt6

If you are a regular user, stick with the Qt5 versions of the release this week. It is the same as before.

Hydrus is moving up to a new version of its UI library, Qt. The new version has a ton of bug fixes and generally better support for newer OS concepts like UI scaling. I am going to be putting out releases for both 5 and 6 for a month or two, testing it with advanced and then normal users, and then will switch to 6 exclusively. Everything seems to be going well, and you don't have to do anything.

If you are an advanced user though, please try out the Qt6 builds. They work exactly the same as the old ones. Just to be careful, I recommend you not try them on your real database first off, and doubly so if you do not have a great backup. I am not worried about database damage, but you never know, and if there are problems, I don't want to give you inconvenience on your main install. Try a fresh extract on your desktop first to make sure it boots ok, and then delete that extract. Then, if you want to try it on your real database, try doing a 'clean install' as here: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs

If you don't do a clean install, you'll still have Qt5 dlls in your install folder and the client will default to the older version (for now). If you are a macOS user, you don't have the concept of 'clean install', so just run the App as normal, but make sure you have a backup of your database first. There's also no Windows Qt6 installer yet. You can check help->about to see what version of Qt is currently running.

So far, the update has been remarkably smooth for me, with very few bugs. A user has been watching the situation for me and kindly provided a patch to deal with the most important syntax changes, so moving over has not been a massive pain in the neck. I've been using it IRL for a few days now and I think things are just that bit smoother and less flickery.

I am particularly interested in Linux and macOS users' feedback. So far, the main limitation I know about is that Windows 7 can't run Qt6 (it is just too old), but there may be other issues in other platforms. Let me know, and we'll see if we can iron them out.

I am going to keep hydrus Qt5 compatible, so anyone who needs to stay on it but wants to keep updating will have the option of running from source.
Open

R: 5 / I: 1 / P: 12

Version 493

https://youtube.com/watch?v=5VygwTBgph4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v493/Hydrus.Network.493.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v493/Hydrus.Network.493.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v493/Hydrus.Network.493.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v493/Hydrus.Network.493.-.Linux.-.Executable.tar.gz

I had a great week working on some fixes and prototype EXIF support.

EXIF

EXIF are metadata labels embedded in some media files, usually JPEG. They might say the make of camera that took a picture, or its aperture/ISO, the GPS coordinates it was taken at, how an image is rotated, or the DPI of a logo for printing purposes. There are many different possible fields. I have wanted to add support to view and even search this data for a long time, and this week we start with something simple but not user-friendly.

The media viewer now has a 'cog' icon on the top hover. On JPEGs, it lets you check for EXIF data. Most files don't have it, but if it does (usually this is photos or professional art exported from Photoshop etc...), it now throws up a little window to see every field.

The duplicate filter now actively scans for EXIF data and says if one or both files have it, just like the recent addition for ICC Profiles. Many websites strip EXIF data on upload, so if you have two exact dupes, the one with EXIF data is probably closer to the 'original' version.

Now I have this framework, I would like to extend it. Beyond general polish like replacing the cog icon with something nicer, and only enable it when I know there is some EXIF to show, I want to cache 'has exif data yes/no' in the database and allow you to search by that. I expect I'll also add the actual EXIF data itself to the database one day, so you'll be able to search all your pictures for iPhone 6 photos or whatever.

So, if you are interested in EXIF, please give this a go and let me know what you think. This feature was taking so long to happen that I decided to just spam out a rough v1.0 that I can keep improving.

full list

- EXIF:
- in the first step of 'official' EXIF support, the media viewer now has a 'cog' button on the top hover, enabled when looking at a jpeg, that will check the file for EXIF data. if found, it will throw it up on a simple new window that shows EXIF id, label, and value. this is a hacked-together prototype, not super user-friendly, but it works. let me know what you think, and please send me any files that have weird EXIF that doesn't parse right but you think should. I already discovered a file with a null character that wouldn't display in UI, that sort of thing
- GPS EXIF values are also parsed and extracted
- made it so you can double-click a row in this new window to copy an EXIF value to clipboard
- in the duplicate filter, if one or both files have exif data, this is now noted in the comparison statements, just like ICC profile! (issue #469)
- obvious future extensions here will be storing 'has exif' in the database and allowing its presence to be searchable and enabling the cog button (or a nicer 'exif' button) only when there is known data to see. a subsequent step would be actually caching the data in the database for full EXIF search
- as a side thing, we're now set up on the hydrus end to pull TIFF EXIF, but PIL doesn't seem to offer it, so we'll have to wait for a different solution there
Open

R: 2 / I: 1 / P: 12

Is there a way to change the upper limit on gifs from 32?
I'm trying to save a 52Mb sized gif and I got prompted with this message.
Open

R: 1 / I: 0 / P: 12

Version 492

https://youtube.com/watch?v=N9LFp_brHvE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v492/Hydrus.Network.492.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v492/Hydrus.Network.492.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v492/Hydrus.Network.492.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v492/Hydrus.Network.492.-.Linux.-.Executable.tar.gz

I had a good week mostly cleaning code and adding some things for advanced users.

highlights

If you are in advanced mode, the file sort and collect controls now have a 'tags' button. This lets you determine which tag service that particular sort/collect applies to. If you are this tag-clever, let me know how it works for you. This tag button is the same thing that autocomplete dropdowns use, and I expect it to soon get a 'multiple location' makeover like the file button did with multiple file services.

The archived file delete lock (under options->files and trash) gets a pass this week. If you try to delete files that are currently locked, it now makes a popup with a button to see those specific files, so you can decide what to do. The duplicate filter also handles the different situations (like 'archive both files' + 'delete the worse file') better.

The duplicate filter also now shows if one or both files have an ICC profile.

The shortcut actions to 'move page selection home/left/right/end' now try to stay on the same level if you hit them several times. If you use these actions, try them out through a mix of page of pages and you'll see how it works now. It remembers the current level within three seconds of the last move event. This was requested by several users, and there isn't a nice way to do it so I hacked an answer, so let me know what you think.

full list

- sort and collect updates:
- for big brain users, the collect control now has a tag domain button. it only shows if you are in advanced mode (issue #572)
- the sort control also has a tag domain button hidden behind advanced mode. it applies to system:num tags and namespace sorting
- the collect control now appears on all import pages
- .
- archived file delete lock:
- the duplicate processing action code now no longer archives files that are due for deletion right before that deletion. this was hitting the archive delete lock
- if archive delete lock is on and the 'other' file in the duplicate filter is archived, the option to 'this is better, delete the other' is now disabled
- if you attempt to delete a delete-locked file during normal browsing, or if an automatic system like export folders wants to but fails on some, a popup is now made with a button to show the files that were filtered out so you can review the situation and fix it if you want
- I am considering adding a dialog to say 'hey, this is locked, want to send back to inbox?' to fix these situations in a nice way, but I think this is probably a bad idea in terms of workflow, design, and my sanity given all the edge cases and potential future expansions of lock rules. maybe I'll add a simple 'delete and override lock checks' option, but a lock is a lock tbh. for now, I will focus on this better UI feedback of currently delete-locked files and make it simpler for humans to remove any locks
Open

R: 3 / I: 0 / P: 12

Version 491

https://youtube.com/watch?v=OQEDWiM-QRI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v491/Hydrus.Network.491.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v491/Hydrus.Network.491.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v491/Hydrus.Network.491.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v491/Hydrus.Network.491.-.Linux.-.Executable.tar.gz

I had a good couple of weeks doing a behind the scenes overhaul and a variety of quality of life work. I was ill last week and put off the release, but I am feeling great now.

metadata import/export

As is often the case, an important overhaul makes few actual changes front end. I've been trying to do this for a while, but now the 'export/import tags in a neighbouring .txt file' routine now works on completely new tech. Rather than hardcoded tags into .txt files, it now uses a modular system that I will be able to expand in future to support filtering and string processing, more metadata types like ratings and URLs, JSON and XML as the file formats, and will even allow funky migrations like converting tags to URLs.

As a side benefit, Export Folders now support tags .txt export.

For now, all the UI front-end looks the same, but please expect this to change in future. I'll be writing some nice unified panels and dialogs to handle the new objects as I write them.

advanced user highlights

The 'OR*' advanced tag input now supports system predicates! It uses the same system predicate parser as the Client API, so you can now type or copy/paste most system predicate text and get something both useful and complicated. The text that parses isn't always exactly the same as the predicate label, so check out the big list of example parseable system predicates here: https://hydrusnetwork.github.io/hydrus/developer_api.html#get_files_search_files

Also, if you are a parser creator, String Processing now has a Tag Filter processing step. Let's say you can grab all the tags from somewhere but you need to filter out a handful of non-tag text like '+' and '?', or you are able to create hydrus namespaced tags and want to filter by namespace, just insert this into your string processing and it should be much easier than messing around with long regexes.

R: 0 / I: 0 / P: 12

Moving Hydrus library between computers

I'm running Hydrus on a Mac, and tried to move the .app of my library (23GB) onto another Mac, but found it didn't work. Is there an easy way to do this?

Thanks!
Open

R: 3 / I: 0 / P: 12

Version 490

https://youtube.com/watch?v=LBzE9JMoCeE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v490/Hydrus.Network.490.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v490/Hydrus.Network.490.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v490/Hydrus.Network.490.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v490/Hydrus.Network.490.-.Linux.-.Executable.tar.gz

I had a good week working on a variety of small jobs.

highlights

While working with a user, I discovered I had recently messed up the initialisation of the image caches, causing them to use default values and be too small for many power users. This is fixed, so I hope you will get some smoother media viewing, especially when it comes to very large images.

I tidied up some weird file service bugs and annoyances that came from the multiple local file services transition. Fixed some weird 'delete from x' entries on the thumbnail menu, stopped spamming 'all my files' in places it wasn't helpful, figured out better service ordering, just some simple workflow stuff.

And I think I have fixed another PTR processing bug some users had. If you have had 'this update file was missing' errors that wouldn't fix themselves, please try again and let the automatic maintenance run one more time--it should repair your records this time.

And just a fun thing--Mr Bones now has more numbers, and more neatly laid out.

full list

- misc:
- fixed a stupid bug that meant the image caches were initialising with default values (as under _speed and memory_) until you opened and OKed the options dialog (or did some other options-refresh events). sorry for the trouble, please enjoy some smoother image browsing.
- mr bones now shows more numbers, and in a neater table. it should be clearer what the percentages are for now, too
- the _manage->regenerate_ thumbnail menu has additional quick maintenance commands for presence and integrity checks and regenerating data in the similar files system
- wrote a new 'special duplicate' button for the edit shortcut set dialog. the list on this dialog doesn't allow duplicates (which meant the old 'duplicate' button was doing nothing), so this duplicates the current actions with 'incremented' shortcut keys. 'a' becomes 'b', 'ctrl+5' becomes 'ctrl+6', and so on. it doesn't always work, but if you want to make ten shortcuts for setting rating 1-10, this should help
- fixed an issue where the thumbnail banner text and the media viewer background text was not changing size or font according to QSS stylesheet rules (issue #1173)
- SIGTERM should now cause a clean program exit (previously it killed the GUI App but left some daemon threads alive for thirty seconds or more). unlike SIGINT, it will not ask you if you are sure you want to exit or if you would like to do shutdown maintenance--it just closes the client promptly
- fixed a bug in last week's importer page status improvements--the hard drive import page wasn't showing all the updates it should have
- brushed up some backup help
- .
- file services:
- fixed a bug where advanced users could set 'all known files'/'all known tags' on a search dropdown. this search domain is not supported
- in the archive/delete filter, if the current location is 'all my files' and the files being deleted are only in one local file domain, the surplus 'all my files' will no longer appear at the top of the filter's commit dialog
- the file services in the thumbnail select/remove menu are now sorted in the same order as the file domain button in search dropdowns
- the thumbnail select/remove menus now exclude 'all my files' and 'all local files' if those choices are redundant (e.g. if you only have files in 'my files', 'all my files' will be hidden)
- fixed some incorrect 'delete from x' actions appearing in thumbnail right-click menus
Open

R: 2 / I: 0 / P: 12

Version 489

https://youtube.com/watch?v=6rboksqjPy4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v489/Hydrus.Network.489.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v489/Hydrus.Network.489.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v489/Hydrus.Network.489.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v489/Hydrus.Network.489.-.Linux.-.Executable.tar.gz

I had a good week getting back into the swing of things. I fixed some important bugs and improved some UI.

highlights

All the downloader pages--gallery, watcher, urls, and simple--have a revamped status system. All the text that shows how file or gallery downloads are going is now generated in a better way, with more error states (e.g. it will tell you when your gallery stopped because it hit the file limit, or when one of the emergency pause states under the network menu has kicked in), and logic in edge cases is improved. Everything is unified now, so the texts are the same across all pages. Also, if a gallery query or watched thread is 'pending', its text now reports that it is waiting for a work slot, rather than staying blank. There _shouldn't_ be any situations now where a downloader is unpaused with work to do but has blank status.

If you use the multiple local file services system, the archive/delete filter now presents more options when you are done. If the files are in more than one local file service, you can choose where you delete them from, including all applicable. This was confusing and opaque before, so I hope this makes it more clear what is happening and gives you more choice.

I _believe_ I have fixed an important bug some users were having with PTR processing. There was an annoying issue about a 'definitions' file being seen as a 'content' file, or vice versa, that the automatic maintenance could not fix. I finally managed to reproduce the issue and fixed it. I schedule a fix in the update this week, so if you have been hit by this, please wait for one more round of file maintenance 'metadata' scans, and then unpause the PTR one more time. Essentially, I think I fixed the automatic maintenance. Let me know how you get on!

full list

- downloader pages:
- greatly improved the status reporting for downloader pages. the way the little text updates on your file and gallery progress are generated and presented is overhauled, and tests are unified across the different downloader pages. you now get specific texts on all possible reasons the queue cannot currently process, such as the emergency pause states under the _network_ menu or specific info like hitting the file limit, and all the code involved here is much cleaner
- the 'working/pending' status, when you have a whole bunch of galleries or watchers wanting to run at the same time, is now calculated more reliably, and the UI will report 'waiting for a work slot' on pending jobs. no more blank pending!
- when you pause mid-job, the 'pausing - status' text is generated is a little neater too
- with luck, we'll also have fewer examples of 64KB of 503 error html spamming the UI
- any critical unhandled errors during importing proper now stop that queue until a client restart and make an appropriate status text and popup (in some situations, they previously could spam every thirty seconds)
- the simple downloader and urls downloader now support the 'delay work until later' error system. actual UI for status reporting on these downloaders remains limited, however
- a bunch of misc downloader page cleanup
Open

R: 1 / I: 0 / P: 13

Version 488

https://youtube.com/watch?v=AQOfIENN2tk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v488d/Hydrus.Network.488d.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v488d/Hydrus.Network.488d.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v488d/Hydrus.Network.488d.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v488d/Hydrus.Network.488d.-.Linux.-.Executable.tar.gz

I had a good simple week making a clean release before my vacation.

Everything is misc this week, nothing earth-shattering, just a bunch of cleanup and little stuff. If you have any wavpack files, try importing them!

full list

- the client now supports 'wavpack' files. these are basically a kind of compressed wav. mpv seems to play them fine too!
- added a new file maintenance action, 'if file is missing, note it in log', which records the metadata about missing files to the database directory but makes no other action
- the 'file is missing/incorrect' file maintenance jobs now also export the files' tags to the database directory, to further help identify them
- simplified the logic behind the 'remove files if they are trashed' option. it should fire off more reliably now, even if you have a weird multiple-domain location for the current page, and still not fire if you are actually looking at the trash
- if you paste an URL into the normal 'urls' downloader page, and it already has that URL and the URL has status 'failed', that existing URL will now be tried again. let's see how this works IRL, maybe it needs an option, maybe this feels natural when it comes up
- the default bandwidth rules are boosted. the client is more efficient these days and doesn't need so many forced breaks on big import lists, and the internet has generally moved on. thanks to the users who helped talk out what the new limits should aim at. if you are an existing user, you can change your current defaults under _network->data->review bandwidth usage and edit rules_--there's even a button to revert your defaults 'back' to these new rules
- now like all its neighbours, the cog icon on the duplicate right-side hover no longer annoyingly steals keyboard focus on a click.
- did some code and logic cleanup around 'delete files', particularly to improve repository update deletes now we have multiple local file services, and in planning for future maintenance in this area
- all the 'yes yes no' dialogs--the ones with multiple yes options--are moved to the newer panel system and will render their size and layout a bit more uniformly
- may have fixed an issue with a very slow to boot client trying to politely wait on the thumbnail cache before it instantiates
- misc UI text rewording and layout flag fixes
- fixed some jank formatting on database migration help

next week

I am now off for a week. I think I need it! I'm going to play a ton of vidya, shitpost the big streams that are happening, fit some Wagner in, and get on top of outstanding IRL stuff. I'll be back to catch up my messages on Saturday the 18th.

Thanks everyone!
Open

R: 2 / I: 0 / P: 13

Version 487

https://youtube.com/watch?v=VKuGYKkH3oA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v487/Hydrus.Network.487.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v487/Hydrus.Network.487.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v487/Hydrus.Network.487.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v487/Hydrus.Network.487.-.Linux.-.Executable.tar.gz

I had an ok week. I was a unexpectedly short on time, so I couldn't get everything I wanted done, but I cleared out some small work.

highlights

The big update last week, which I recommended only for advanced users, went well. There don't seem to be any obvious problems with the logic of the new search cache, so I now recommend it for everyone. You will be presented with a popup just before the update runs, giving you an estimate of how long it thinks it will take. Most users should take 5-10 minutes, but if you have millions of files, it will be longer. Just let it run and some things will run a bit faster and neater in the background. If you have played with 'multiple local file services', then check out the new 'all my files' domain you will see--this is basically an efficient umbrella of all your local file services. It works super fast for things like the duplicates system.

I also put some time into the duplicate filter this week. The logic of the queue is improved again, so some rare errors when reaching the end of a batch should be fixed. I also integrated manual file deletes into the queue processing: now, when you manually delete a file, or both, the deletes will not happen until you commit--just like the other decisions you are making--and they are undoable if you select 'forget' or go back a pair. You also won't see a file you manually deleted again in a batch (it'll auto-skip if that file comes up again).

Also, the duplicate filter now has a little 'send pair to page' button, which publishes the current pair to the duplicates page that made the filter, just in case you want to save them for some extra processing after you are done filtering. You can do this with multiple pairs and they'll just stack up in the page.

A couple other neat things happened in last week's advanced-user-only release, which I will repeat here:

The 'media viewers' shortcut set has three new zoom actions: 'switch between 100% and max', 'switch between canvas and max', and 'zoom to max'.

When you enter pairs in the tag sibling dialog, it shouldn't complain about loops anymore, but instead automatically break them, just like how it will auto-petition an A->B, A->C conflict.
Open

R: 3 / I: 0 / P: 13

Version 486

https://youtube.com/watch?v=ZUrcYKghr-Y [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v486/Hydrus.Network.486.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v486/Hydrus.Network.486.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v486/Hydrus.Network.486.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v486/Hydrus.Network.486.-.Linux.-.Executable.tar.gz

I had a great week working on a variety of smaller issues and some important database updates.

The release this week is only recommended for advanced users. I make an important change, and I want to make sure the update works quickly and without problems before I roll it out to everyone. If you are not an advanced user, please check back in next week!

The update will also take a few minutes this week.

all my files

So, I have made a new virtual service, 'all my files', which covers the union of all your local file services. This service is very similar to 'all local files', but it does not include trash or repository files. It provides a bunch of tools across the program for quick and precise searching of all the files that have value and are worth looking at.

When you update, this new service will be created and populated. It will take a few minutes, longer if you have millions of files and tags. My 2.8-million-file ptr-syncing client took 32 minutes. There are progress updates on the splash window.

Once you are booted, you will see 'all my files' in review services and the file domain selector if you have more than one local file domain. Feel free to play around with it--it will run a lot faster than previously going 'multiple locations' and unioning all your local file services.

The code is working really well on my end, and I am not afraid of anything being damaged, but if something goes wrong, it may require some clever/slow regeneration to fix. The main things I would like to know are:

1) Did your update take significantly longer than ~100k files/minute? Did it get held up on anything?
2) After some use, have you noticed any file/tag miscounting with 'all my files'?

As always, make a backup before you update.

other highlights

The 'media viewers' shortcut set has three new zoom actions: 'switch between 100% and max', 'switch between canvas and max', and 'zoom to max'.

When you enter pairs in the tag sibling dialog, it shouldn't complain about loops anymore, but instead automatically break them, just like how it will auto-petition an A->B, A->C conflict.

The database now cleans up after itself more thoroughly. Some users have been having trouble with very large 'WAL' files, some getting to be multiple GB, and perhaps seeing bloated memory use along with it. A set of new maintenance routines now force write-flushing at regular intervals. In my testing, there is no lag related to this, but I will be interested to hear if anyone gets new commit hang-ups during very heavy work. If you have had a huge WAL, let me know if this helps!
Open

R: 10 / I: 1 / P: 13

Version 484

https://youtube.com/watch?v=OtPsKtUyGxg [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v484/Hydrus.Network.484.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v484/Hydrus.Network.484.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v484/Hydrus.Network.484.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v484/Hydrus.Network.484.-.Linux.-.Executable.tar.gz

I had an ok week. I fixed some things, improved some quality of life, and made internal file migration a bit easier.

highlights

Last week's debut of multiple local file services went well! As far as I know, no one who tried it out had any big problems, and my main concerns--mostly that it needs some better migration tools and workflows and 'this file is in here' UI--proved true. So, I know what I have to do and will keep working. Multiple local file services remains for advanced users for now, but I hope to launch it properly for everyone, with nice help, next week.

However, while doing this work, I did accidentally break the simple version of the 'delete files' dialog when files were in the trash--rather than say 'delete these permanently?', it just wouldn't appear. This was due to a logical oversight where it wasn't testing and counting up 'trash' status correctly. It is fixed now.

Also, there was a problem with the new duplicate filter queue for users who have done a good bit of processing. A certain function that in complicated situations automatically skips some pairs was failing whenever it hit the end of a batch. This is also fixed now, thank you for the great reports on this.

For multiple local file services, I updated the UI code, fixing some little bugs and improving the workflow when you have complicated situations, and I integrated the shortcuts system and the media viewer. You can now create 'add/move to service x' actions in the 'media' shortcut set, and the media viewer has the same add/move menu on right-clicks.

The media viewer has several other improvements: I think I fixed that annoying bug where a fullscreen borderless view of media that exactly fits the screen would sometimes not resize when you went back to normal window mode! Also, scrolling the mouse over the taglist hover window should no longer ever cause a 'previous/next media' event. And I have implemented a 'short and simple' version of the video/audio scanbar to show (instead of completely hiding it) when your mouse is away--just a few pixels to show things 'at a glance'. Even though it covers a few pixels of video at the bottom, I liked this so much that I set it as the default for all users. If you don't like it, you can hide it again with the new setting under options->media.
Open

R: 1 / I: 0 / P: 13

Version 485

https://youtube.com/watch?v=AKgjOCuW_MU [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v485a/Hydrus.Network.485a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v485a/Hydrus.Network.485a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v485a/Hydrus.Network.485a.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v485a/Hydrus.Network.485a.-.Linux.-.Executable.tar.gz

I had a good week. The multiple local file services system is now ready for all users.

multiple local file services

I have written some proper help for this new system to talk about what it is and how to use it. The basic idea is you can now have more than one 'my files', which lets you compartmentalise things for privacy or workflow reasons. The help is here:

https://hydrusnetwork.github.io/hydrus/advanced_multiple_local_file_services.html

All users can try this out--you no longer have to be in advanced mode--but in terms of experience level, I recommend it to people who are at least comfortable with tag siblings and parents.

This system is fundamentally feature complete. The outstanding immediate problems are that it file location doesn't show up in UI very well yet, the Client API should plug into it better, and it needs some en masse controls to do large file migrations and client merging. I hope to work on these in the coming weeks.

If you give this a go, let me know what you think!
Open

R: 4 / I: 0 / P: 13

Version 483

https://youtube.com/watch?v=ymI1g2VjyCY [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v483/Hydrus.Network.483.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v483/Hydrus.Network.483.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v483/Hydrus.Network.483.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v483/Hydrus.Network.483.-.Linux.-.Executable.tar.gz

I had a good couple of weeks doing some regular work and getting 'multiple local file services' ready for testing.

multiple local file services

This is not ready for everyone yet! Advanced users only for now please.

I turned multiple local file services on in debug mode last week, just to see how things were looking, and it turned out suprisingly great, no big problems. For several months now I have been doing prep work for it, and that seems to have paid off. I decided to finish the last important things and get a v1.0 out.

So, it is now possible to have multiple 'my files' services in your client, and to search, import to, and migrate files between them. These services are completely blind to each other, so searching for autocomplete tags in one will not return suggestions from another. The hope is this will allow fairly good sfw/nsfw-style separations in clients and open up interesting new contained workflows.

I am recommending this only for advanced users for now, and moreso only those who have been following this feature. I have not yet written up nice help for this, and some of the UI/workflow is still not user friendly, so what I would like is for people who are enthusiastic to try it out and let me know what they think. I really haven't run into any massive errors, but I won't encourage you to go crazy on a real client yet. Go nuts on a new empty test client, or experiment carefully on a real client, just in case something goes wrong, and I will keep polishing the experience.

The basics are: you can now make a new 'local file domain' in manage services. file import options now lets you import to different or multiple local file domains, and thumbnail right-click lets you copy or move files between them too. The normal search page dropdown lets you jump between local services just like searching trash, and of course it now supports multiple domains if you want to do a union. The delete and undelete commands are similarly a little more powerful when you start adding new services. Check out the changelog for more specific details.

Next step I think is to make it more obvious when thumbnails/files are in certain services, since at the moment you have to scan the text on the status bar, top media hover, or thumbnail menu. Maybe custom icon rules (e.g. 'when the file is in "sfw" domain, give it a flower icon'). Then general polish like shortcut integration, maybe some more search tech, and then I really want to write a nice help document for it all to introduce normal experienced users to the idea, and some 'merge these clients' tech would be great, so users who have been using two or more clients for years can finally combine them into one.
Open

R: 3 / I: 0 / P: 13

Version 482

https://youtube.com/watch?v=PGEZutQ-tCM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v482/Hydrus.Network.482.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v482/Hydrus.Network.482.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v482/Hydrus.Network.482.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v482/Hydrus.Network.482.-.Linux.-.Executable.tar.gz

I had a great week doing cleanup and other simple work.

highlights

I fixed the problem where clicks on a scrolled taglist were going to the wrong location. I was cleaning up some ancient wx->Qt code hacks and it seems I rarely scroll and click when working, so I never noticed the problem. I have a new test to make sure this does not happen again. Sorry for the trouble!

The URLs in the top-right hover menu are now styled better. No longer underlined, and now colourable by QSS. I have updated all the default stylesheets that come with the client (you can set these under options->style) to have some decent colours. If you have your own custom QSS, check my default to see how to set it yourself.

You can now set duplicate action options to 'always archive both files', if you want to play with making the duplicate filter do some of the work of the archive/delete filter.

Also, the duplicate filter now has improved image prefetch. There should be less flickering when you switch from A to B the first time and when you action a pair and move to the next. Please note that if you still get flicker for 4k images, try boosting the image cache size under options->speed and memory (I boosted the default up to 384MB this week, so you might like to give it some more too).

full list

- misc:
- fixed the stupid taglist scrolled-click position problem--sorry! I have a new specific weekly test for this, so it shouldn't happen again (issue #1120)
- I made it so middle-clicking on a tag list does a select event again
- the duplicate action options now let you say to archive both files regardless of their current archive status (issue #472)
- the duplicate filter is now hooked into the media prefetch system. as soon as 'A' is displayed, the 'B' file will now be queued to be loaded, so with luck you will see very little flicker on the first transition from A->B.
- I updated the duplicate filter's queue to store more information and added the next pair to the new prefetch queue, so when you action a pair, the A of the next pair should also load up quickly
- boosted the default sizes of the thumbnail and image caches up to 32MB and 384MB (from 25/150) and gave them nicer 'bytes quantity' widgets in the options panel
- when popup windows show network jobs, they now have delayed hide. with luck, this will make subscriptions more stable in height, less flickering as jobs are loaded and unloaded
- reduced the extremes of the new auto-throttled pending upload. it will now change speed slower, on less strict of a schedule, and won't go as fast or slow max
- the text colour of hyperlinks across the program, most significantly in the top-right media hover window, can now be customised in QSS. I have set some ok defaults for all the QSS styles that come with the client, if you have a custom QSS, check out my default to see what you need to do. also hyperlinks are no longer underlined and you can't 'select' their text with the mouse any more (this was a weird rich-text flag)
- the client api and local booru now have a checkbox in their manage services panel for 'normie-friendly welcome page', which switches the default ascii art for an alternate
- fixed an issue with the hydrus server not explicitly saying it is utf-8 when rendering html
- may have fixed some issues with autocomplete dropdowns getting hung up in the wrong position and not fixing themselves until parent resize event or similar
Open

R: 3 / I: 0 / P: 13

Version 481

https://youtube.com/watch?v=nShSEUBKe3o [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v481/Hydrus.Network.481.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v481/Hydrus.Network.481.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v481/Hydrus.Network.481.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v481/Hydrus.Network.481.-.Linux.-.Executable.tar.gz

I had a great week. Lots of different small jobs done.

notes and hover windows

I'm happy with last week's work making notes show in media viewers, but I introduced some little bugs while rewriting hover windows. I have now fixed the bad text colour behind the top hover, the problem where clicking on tags or greyspace was propagating up to the archive/delete and duplicate filters, the bad hover panel colour on non-default stylesheets, and some note window position and size issues.

Also, for notes, you can now right-click them to collapse them in the hover window. Right-click again on the name to expand again. This is a test, really, just to see if it helps navigating files with many long notes. Double-clicking on the note tab in the edit dialog lets you rename, and a checkbox under the new options->notes now lets you choose whether the text caret starts at the beginning or end of the document when editing.

Furthermore, I have updated all the icon buttons in all the hovers to no longer take focus when you click on them. They were previously stealing arrow key and space after a click (to do button-to-button form navigation), which meant you couldn't click on, say, a duplicate filter action button and then go back to arrow keys to navigate. Now you should be able to mix clicks and arrow keys without trickery. If this affects you, let me know how it goes!

other highlights

If you didn't like the recent 'ctrl- and shift-clicks no longer show files in the preview viewer' change, check out the new checkboxes under options->gui pages. You can make either click type focus for all files again or just files with no duration--if you don't want noisy videos being annoying while you ctrl-click.

The 'advanced mode' autocomplete dropdown now has two 'OR' buttons. The left one opens a new empty OR edit dialog, the right one opens the advanced text parsing input as before.

full list

- fixes and improvements after last week's hover and note work:
- fixed the text colour behind the top middle hover window
- stopped clicks on the taglist and hover greyspace being duplicated up to the main canvas (this affected the archive/delete and duplicate filter shortcuts)
- fixed the background colour of the hover windows when using non-default stylesheets
- fixed the notes hover window--after having shown some notes--could then lurk in the top-left corner when it should have been hidden completely
Open

R: 2 / I: 0 / P: 13

Version 480

https://youtube.com/watch?v=R1t6iNG28zI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.Linux.-.Executable.tar.gz

I had a good week. Notes now display on the media viewer.

notes

Notes have always been a slightly hidden system, a bit like ratings were. Today is a step forward to exposing them. Any file that has notes (you can start adding notes to a file by hitting manage->notes on their right-click menu) will now show them in the media viewer, just below the top-right hover window. They get their own hover window too, if you mouse over them. If you click on a particular note, the 'edit notes' dialog opens on it.

This is a first version, and a little ugly, but I'm happy we now have something I can iterate on in future. If you are a big notes person, please let me know how it works best and worst for you. If you have unusual font style, size, or colour, let me know if it goes crazy or sizes too short or tall.

While working on this, I rewrote the media viewer's hover windows to be more sensible, something I have been planning for a long time. They are now 'embedded' into the parent canvas, which should reduce a variety of jank behaviour--particularly, if you now click a hover, the main media viewer window no longer loses focus. There is still some hackery in the system to clean up, but I hope it'll work better overall for you.

Unfortunately, I just did not get to note merge in the duplicates system or note parsing. That'll have to be for the future.

the rest

A user is working on a neat 'gallery share' system that plugs into the Client API, here: https://github.com/floogulinc/hyshare . It looks like a great replacement for my old 'local booru', so if you are interested in sharing groups of files straight from your client with friends over an attractive booru-like interface, check it out!

I copied the 'file log' and 'search log' button menus, where you can do en masse actions like 'retry all failed' and 'export all to clipboard', to both the log review panels and the downloader/watcher list right-click menus. It is now possible to big actions on logs without highlighting anything.

Just a small thing, but when you select a gallery in the gallery downloader page, the focus moves straight to the query text input, so you can start typing immediately.
Open

R: 3 / I: 0 / P: 13

Version 479

https://youtube.com/watch?v=P7MsTw9s03o [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v479a/Hydrus.Network.479a.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v479a/Hydrus.Network.479a.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v479a/Hydrus.Network.479a.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v479a/Hydrus.Network.479a.-.Linux.-.Executable.tar.gz

I had an ok week doing a mix of work.

highlights

I made it so when you shift-select some thumbnails, you can now move 'back' to deselect what you just selected. This also remembers what was previously selected before the shift-select started, so it works basically like an undo. I like how this works, but as part of it I had to make every thumbnail 'hit' focus in the preview viewer, which is not how all selects worked before. I already find this annoying, so I think I am going to make the system more clever and add some options around this behaviour.

I think I improved the duplicate filter's zoom locking, particularly when one of a pair is portrait and the other is landscape. It should generally be more 'stable' now, but let me know if you still have any trouble.

WebPs should now show transparency correctly!

The new file history chart has another round of better math, an 'archived files' line, and you can hide the deleted files line if it is too big.

I reworked a little help to make some 'ok, I know the basics, what next?' things clearer to find. If you missed learning about the autocomplete dropdown, tag wildcards, or OR searching, please check here: https://hydrusnetwork.github.io/hydrus/getting_started_searching.html

full list

- misc:
- when shift-selecting some thumbnails, you can now reverse the direction of the select and what you just selected will be deselected, basically a full undo (issue #1105)
- when ctrl-selecting thumbnails, if you add to the selection, the file you click is now focused and always previewed (previously this only happened if there was no focused file already). this is related to the shift-select logic above, but it may be annoying when making a big ctrl-selection of videos etc.. so let me know and I can make this more clever if needed
- added file sort 'file->hash', which sorts pseudorandomly but repeatably. it sounds not super clever, but it will be useful for certain comparison operations across clients
Open

R: 0 / I: 0 / P: 14

Nijie working?

Does Nijie work for anyone else anymore? I get 503 errors every time, no matter the user-agent or cookies I use.
Open

R: 2 / I: 0 / P: 14

Version 478

https://youtube.com/watch?v=eGybwV3U9W8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v478/Hydrus.Network.478.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v478/Hydrus.Network.478.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v478/Hydrus.Network.478.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v478/Hydrus.Network.478.-.Linux.-.Executable.tar.gz

I had a good week mostly fixing some bugs and cleaning things up behind the scenes. There's nothing super big to highlight, but I did improve the new file history chart (help->view file history). The axes are a bit nicer, and I fixed a small counting logic bug in the 'inbox' line.

full list

- misc:
- if a file note text is crazy and can't be displayed, this is now handled and the best visual approximation is displayed (and saved back on ok) instead
- fixed an error in the cloudflare problem detection calls for the newer versions of cloudscraper (>=1.2.60) while maintaining support for the older versions. fingers crossed, we also shouldn't repeat this specific error if they refactor again
- .
- file history chart updates:
- fixed the 'inbox' line in file history, which has to be calculated in an odd way and was not counting on file imports adding to the inbox
- the file history chart now expands its y axis range to show all data even if deleted_files is huge. we'll see how nice this actually is IRL
- bumped the file history resolution up from 1,000 to 2,000 steps
- the y axis _should_ now show localised numbers, 5,000 instead of 5000, but the method by which this occurs involves fox tongues and the breath of a slighted widow, so it may just not work for some machines
- .
- cleanup, mostly file location stuff:
- I believe I have replaced all the remaining surplus static 'my files' references with code compatible with multiple local file services. when I add the capability to create new local file services, there now won't be a problem trying to display thumbnails or generate menu actions etc... if they aren't in 'my files'
- pulled the autocomplete dropdown file domain button code out to its own class and refactored it and the multiple location context panel to their own file
- added a 'default file location' option to 'files and trash' page, and a bunch of dialogs (e.g. the search panel when you make a new export folder) and similar now pull it to initialise. for most users this will stay 'my files' forever, but when we hit multiple local file services, it may want to change
- the file domain override options in 'manage tag display and search' now work on the new location system and support multple file services
- in downloaders, when highlighting, a database job that does the 'show files' filter (e.g. to include those in trash or not) now works on the new location context system and will handle files that will be imported to places other than my files
- refactored client api file service parsing
- refactored client api hashes parsing
- cleaned a whole heap of misc location code
- cleaned misc basic code across hydrus and client constant files
- gave 'you don't want the server' help page a very quick pass
- .
- client api:
- in prep for multiple local file services, delete_files now takes an optional file_service_key or file_service_name. by default, it now deletes from all appropriate local services, so behaviour is unchanged from before without the parameter if you just want to delete m8
- undelete files is the same. when we have multiple local file services, an undelete without a file service will undelete to all locations that have a delete record
- delete_files also now takes an optional 'reason' parameter
Open

R: 3 / I: 1 / P: 14

Version 477

https://youtube.com/watch?v=mVG77xTPH6E [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v477/Hydrus.Network.477.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v477/Hydrus.Network.477.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v477/Hydrus.Network.477.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v477/Hydrus.Network.477.-.Linux.-.Executable.tar.gz

I had a good week. There is a mix of small work, an expansion to the Client API, and a bit of fun.

misc

The network engine now pulls source time directly from file downloads if the server provides a date. This means a whole bunch of sites that haven't provided a good source time until now suddenly do, which improves the new aggregate modified time and also subscription and watcher check timings.

With our new apng parsing tech, I fixed up apng duration parsing, which was until now relying on a fallback default of 24 fps if ffmpeg couldn't figure it out. All apngs are scheduled for another scan.

I fixed an important precedence bug in the network engine that matches URLs to URL Classes. If you have been making a downloader and had Gallery URLs matching as Post URLs, please give it another go. Sorry for the trouble!

Client API

A user has written a cool expansion to the Client API, which I appreciate. You can now fetch, set, and delete file notes! If you are an API dev, check out the documentation for the new calls (fetching notes is now a parameter on file_metadata).

He also made technical improvements. The Client API now supports far longer GET requests, up to 2MB of URL if needed, and the whole API has tentative and experimental support for CBOR instead of JSON if you wish.

file history chart

Another user has for some time been playing around with drawing charts of a client's file history in matplotlib using raw database data. You may have run his script yourself. We have been talking for a while about integrating this into hydrus, and this week I finally got around to implementing it in QtCharts.

Please hit 'view file history' on the help menu to see the new chart. This is a simple, first attempt on my end, but it should show you a cool history of how many files you have had. If you have been using the client for any time, the lines for deleted files and inbox will be very incomplete, but this data will fill out in time.

This was fun to do, and I learned a bit more about QtCharts. I fixed a couple of ugly things in the bandwidth bar chart I made before, and I think I'll do some more here too. I have a thought to start drawing some of our other data, let's say file size or number of file views, and seeing if pareto or normal distributions pop out. Anyway, let me know what you think, and feel free to share your file history chart!
Open

R: 3 / I: 0 / P: 14

Version 476

https://youtube.com/watch?v=tigTaObQORM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v476/Hydrus.Network.476.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v476/Hydrus.Network.476.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v476/Hydrus.Network.476.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v476/Hydrus.Network.476.-.Linux.-.Executable.tar.gz

I had a good week integrating two new features: autocomplete tag search in the client api, and saved 'post times' from downloaders.

post times

'Modified time' is neat, but it isn't super useful for downloaded files--since the file was only just added to your hard drive, it'll always be the same as import time. This week I integrate the 'source time' we parse from various websites to improve the modified time for downloaded files. The objective is to make 'modified' a fairly decent 'this file was completed around this time' number for searching and sorting purposes.

I'm being careful not to overwrite anything. The client now saves its best 'source time' for every different site it downloads from and then the earliest of those + modified date is used as the aggregate modified date. You don't have to do anything, but with luck you will see your new watcher and gallery files start to get some nicer modified times in the media viewer and thumbnail right-click menu.

There are many potential future expansions here. I can grab better post times from sites, show and edit every stored timestamp in UI, allow clever search and sort of those specifically (e.g. 'sort all these files by their danbooru post time), and most importantly make some sort of maintenance system to retroactively fetch a good post time for all the files we downloaded before post times were saved. This is just a first step.

I integrated the new 'archive time' too during this work. This now shows in the media viewer and thumbnail right-click menu similarly and can be sorted by. Search will come soon. I also want to think about optionally filling in some estimate dummy data here for all the files we archived before timestamps were tracked.

client api autocomplete search

A user has helpfully written Client API routines for autocomplete tag search, which is something I have had trouble fitting in. I appreciate the work. This should let the various tools that use the Client API do more tag browsing in future.

The documentation is here: https://hydrusnetwork.github.io/hydrus/developer_api.html#add_tags_search_tags

There are several ways to expand this too, so if you are an API dev interested in it, let me know how it goes.
Open

R: 3 / I: 0 / P: 14

Version 475

https://youtube.com/watch?v=rNFLCB_T2hA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v475/Hydrus.Network.475.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v475/Hydrus.Network.475.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v475/Hydrus.Network.475.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v475/Hydrus.Network.475.-.Linux.-.Executable.tar.gz

I had a good couple of weeks. There's a long changelog of small items and some new help.

new help

A user has converted all my old handcoded help html to template markup and now the help is automatically built with MkDocs. It now looks nicer for more situations, has automatically generated tables of contents, a darkmode, and even in-built search.

It has been live for a week here:

https://hydrusnetwork.github.io/hydrus/

And is with v475 rolled into the builds too, so you'll have it on your hard disk. Users who run from source will need to build it themselves if they want the local copy, but it is real easy, just one line you can fold into an update script:

https://hydrusnetwork.github.io/hydrus/about_docs.html

I am happy with how this turned out and am very thankful to the user who put the work in to make the migration. It all converted to this new format without any big problems.

misc highlights

I queue up some more files for metadata rescans, hopefully fixing some more apngs and figuring out some audio-only mp4s.

System:hash now supports 'is not', so if you want to paste a ton of hashes you can now say 'but not any of these specific files'.

Searches with lots of -negated tags should be a good bit faster now.

I fixed a bug that was stopping duplicate pages from saving changes to their search.

pycharm

I moved to a new IDE (the software that you use to program with) this week, moving from a jank old WingIDE environment to new PyCharm. It took a bit of time to get familiar with it, so the first week was mostly me doing simple code cleanup to learn the shortcuts and so on, but I am overall very happy with it. It is very powerful and customisable, and it can handle a variety of new tech better.

It might be another few weeks before I am 100% productivity with it, but I am now more ready to move to python 3.9 and Qt 6 later in the year.

full list

- new help docs:
- the hydrus help is now built from markup using MkDocs! it now looks nicer and has search and automatically generated tables of contents and so on. please check it out. a user converted _all_ my old handwritten html to markup and figured out a migration process. thank you very much to this user.
- the help has pretty much the same structure, but online it has moved up a directory from https://hydrusnetwork.github.io/hydrus/help to https://hydrusnetwork.github.io/hydrus. all the old links should redirect in any case, so it isn't a big deal, but I have updated the various places in the program and my social media that have direct links. let me know if you have any trouble
- if you run from source and want a local copy of the help, you can build your own as here: https://hydrusnetwork.github.io/hydrus/about_docs.html . it is super simple, it just takes one extra step. Or just download and extract one of the archive builds
- if you run from source, hit _help->open help_, and don't have help built, the client now gives you a dialog to open the online help or see the guide to build your help
- the help got another round of updates in the second week, some fixed URLs and things and the start of the integration of the 'simple help' written by a user
- I added a screenshot and a bit more text to the 'backing up' help to show how to set up FreeFileSync for a good simple backup
Open

R: 6 / I: 0 / P: 14

Version 474

https://youtube.com/watch?v=JYGb9HRCCyg [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v474/Hydrus.Network.474.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v474/Hydrus.Network.474.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v474/Hydrus.Network.474.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v474/Hydrus.Network.474.-.Linux.-.Executable.tar.gz

I had a great week working on small fixes and improvements. There's nothing earth-shattering here to highlight, but just a mix of little work.

full list

- command palette:
- the guy who put the command pallete together has fixed a 'show palette' bug some people encountered (issue #1060)
- he also added mouse support!
- he added support to show checkable menu items too, and I integrated this for the menubar (lightning bolt icon) items
- I added a line to the default QSS that I think fixes the odd icon/text background colours some users saw in the command palette
- .
- misc:
- file archive times are now recorded in the background. there's no load/search/sort yet, but this will be added in future
- under 'manage shortcuts', there is a new checkbox to rename left- and right-click to primary- and secondary- in the shortcuts UI. if you have a flipped mouse or any other odd situation, try it out
- if a file storage location does not have enough free disk space for a file, or if it just has <100MB generally, the client now throws up a popup to say what happened specifically with instructions to shut down and fix now and automatically pauses subscriptions, paged file import queues, and import folders. this test occurs before the attempt to copy the file into place. free space isn't actually checked over and over, it is cached for up to an hour depending on the last free space amount
- this 'paused all regular imports' mode is also now fired any time any simple file-add action fails to copy. at this stage, we are talking 'device disconnected' and 'device failed' style errors, so might as well pause everything just to be careful
- when the downloader hits a post url that spawns several subsidiary downloads (for instance on pixiv and artstation when you have a multi-file post), the status of that parent post is now 'completed', a new status to represent 'good, but not direct file'. new download queues will then present '3N' and '3 successful' summary counts that actually correspond to number of files rather than number of successful items
- pages now give a concise 'summary name' of 'name - num_files - import progress' (it also eli...des for longer names) for menus and the new command palette, which unlike the older status-bar-based strings are always available and will stop clients with many pages becoming multi-wide-column-menu-hell
- improved apng parsing. hydrus can now detect that pngs are actually apngs for (hopefully) all types of valid apng. it turns out some weird apngs have some additional header data, but I wrote a new chunk parser that should figure it all out
- with luck, users who have window focus issues when closing a child window (e.g. close review services, the main gui does not get focus back), should now see that happen (issue #1063). this may need some more work, so let me know
- the session weight count in the 'pages' menu now updates on any add thumbs, remove thumbs, or thumbnail panel swap. this _should_ be fast all the time, and buffer nicely if it is ever overwhelmed, but let me know if you have a madlad session and get significant new lag when you watch a downloader bring in new files
Open

R: 7 / I: 0 / P: 14

Version 473

https://youtube.com/watch?v=F6OR66gZY54 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v473/Hydrus.Network.473.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v473/Hydrus.Network.473.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v473/Hydrus.Network.473.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v473/Hydrus.Network.473.-.Linux.-.Executable.tar.gz

I had a mixed week. Unfortunately some IRL reduced my work time. There's a neat new widget to play with though!

command palette

A user has written a cool widget that helps you navigate the program by keyboard. I have integrated the first version and am interested in feedback. If you have nothing on Ctrl+P for your 'main window' shortcut set, you should get it mapped to that on update.

So, hit Ctrl+P and you'll get a palette where you can type and press up/down and enter to quickly navigate to any of the pages currently open. If you are an advanced mode user, you will also search all of the menubar actions and also the current thumbnail selection menu. This latter part is unfiltered at the moment--you'll just see everything--so be careful. The system needs more polish, including filtering out these more advanced database routines, and proper display for checkbox items 'check' status, and so on.

I can do a lot more with this widget, so give it a go and let me know what you think. I think some of the labels can probably be improved, and I am sure some would like to customise it a little. If you don't like Ctrl+P, just hit up file->shortcuts->the main window and re-map it!

full list

- misc:
- fixed the recent problem with drag and dropping thumbnails to a level below the top row of pages. sorry for the trouble!
- fixed a bug where the client would not load results sorting by 'import time' when the search file domain was a single deleted file domain
- fixed a list display bug in the edit page parser dialog when a subsidiary page parser has two complicated string-match based content parsers
- collections now sort by modified time, using the largest known modified time in their collection
- added sqlite3.exe console back into the windows build--sorry, it was missing since the github build changeover!
- added a note to the help about backing up when tight on space, which I will repeat here: the sqlite database files are very compressible (70GB->17GB on default 7zip settings!), so if you need more space on your backup drive, this is a good way to reclaim it
- .
- command palette:
- a user has written a cool 'command palette' for the program! it brings up a type-and-search interface to navigate to pages or menu entries.
- I have integrated his first version and set the default shortcut to Ctrl+P. users who update will get this shortcut if they have nothing else on Ctrl+P on 'main window' set. if you prefer Ctrl+K or anything else, you can change it under _file->shortcuts->the main window_
- regular users will get a page list they can search and select, advanced users will also get the (potentially dangerous) full scan of the menubar and current thumbnail right-click menu. I will be polishing this latter feature in future to filter out big maintenance jobs and show checkbox status and similar, so if you are advanced, please be careful for now
- try it out, and let me know how it goes. the underlying widget is neat, and I can change its behaviour and extend it significantly
Open

R: 9 / I: 2 / P: 14

Version 472

https://youtube.com/watch?v=tGgmv1yEI-U [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v472b/Hydrus.Network.472b.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v472b/Hydrus.Network.472b.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v472b/Hydrus.Network.472b.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v472b/Hydrus.Network.472b.-.Linux.-.Executable.tar.gz

I had a good week. There are some neat new UI features and several bug fixes.

new stuff

Although it isn't super useful yet, searching in multiple file domains is now complete. If you click the file domain button on a search page's tag autocomplete dropdown, there is now a 'multiple locations' entry. This lets you select a union of several domains. New users will probably only see 'my files' and 'trash', but advanced users will see more, and deleted domains too. Please feel free to play with this. Autocomplete counts should have accurate ranges and the union file searches, limited though they currently are, do work. We are a step closer to fully supporting multiple 'my files' services.

Under options->thumbnails, there is now a dropdown for 'thumbnail scaling'. The default remains 'scale down only', but you can now 'scale to fit' (which scales small things up) or 'scale to fill', which crops and scales up so the thumbnail fills the whole thumbnail space. Give it a go--the animation as one format changes to another is accidentally one of the coolest things I have done.

If you right-click on the page tab bar, there is now a 'pages' submenu, listing all the pages at or below that level, that lets you quickly navigate to them. This is a prototype, basically a copy of the same thing you'll see in a web browser, so let me know what you think--I expect to put more time into it.

bug fixes

I fixed a stupid issue where search pages were refreshing on session load. I regret this slipped through (again!), so I made an explicit test to catch it in future. Sorry for the trouble!

I also fixed an issue with 'do work on shutdown' cancelling repository processing and a couple of other jobs as soon as they started. The 'fast shutdown' logic was working a little too well!

Images with an alpha channel are now tested on load--if the alpha channel is completely opaque, it is stripped. This saves a little memory and CPU, and it means they will be correctly detected as pixel duplicates of their non-alpha-channel'd equivalents.
Open

R: 3 / I: 0 / P: 14

Version 471

https://youtube.com/watch?v=HSdNxvFCZj4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v471/Hydrus.Network.471.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v471/Hydrus.Network.471.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v471/Hydrus.Network.471.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v471/Hydrus.Network.471.-.Linux.-.Executable.tar.gz

I had a great week. The program can now track the last time you viewed a file.

last viewed time

By default, file viewing statistics are on. You can control how they work and turn them off under options->file viewing statistics and if you like clear all your records under the database menu.

If you have them on, hydrus now records the last time you saw a file in the media viewer and preview viewer. A 'last viewed' here counts with the same rules as recording view time (typically you need to look at the file a few seconds), so if you just scroll through ten files in a second, those won't be recorded as views.

You can see the last viewed time in the thumbnail right-click menu with the other view time numbers. You can also search and sort it, like with import or modified time.

These three 'time' sorts are now grouped into a new 'time' entry in the sort menu, and for searching under a new 'system:time' system predicate.

Let me know how this works for you! The search and sort uses the media viewer number for now, but if you are a preview or combined person and this does not work for you, let me know how you would like your situation supported.

other highlights

I added some new ways to import the downloader-pngs you can drop onto network->downloaders->import downloaders and other places. The program is now ok with different versions of this data, so you can now drag and drop an image from your browser onto Lain and it should still work. You can also do 'copy image' and click the new paste button. Should just be a bit more convenient when you only need to use the file once. I hope to add more export methods too (e.g. export to clipboard bitmap, which you can then paste into a site/discord), so it should be possible soon for downloader makers to share their work direct to a person without either ever touching a file.

For advanced users: The colours of the scanbar I put below videos is now editable in QSS! This is a simple test for now, as this was more complicated than you'd think. You should now be able to set the base, border, and nub colour. Check out the example in default_hydrus.qss and give it a go in your own custom QSS files. I expect to move all the other hardcoded colours except for probably tag namespaces and maybe some custom thumbnail borders to QSS in future.

R: 0 / I: 0 / P: 14

Parser Sharing Thread

Can't see one here, maybe I'm blind, I'll start.
Open

R: 3 / I: 0 / P: 15

Version 470

https://youtube.com/watch?v=_SSPdeoAHsM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v470b/Hydrus.Network.470b.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v470b/Hydrus.Network.470b.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v470b/Hydrus.Network.470b.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v470b/Hydrus.Network.470b.-.Linux.-.Executable.tar.gz

I had a great week. The big rewrites are finally doing something interesting, and I got some other stuff done as well.

cleverer search

First of all, I finished my file domain rewrites. File search, tag search, and all the UI that displays it now works on n file domains at once rather than just one. Also, any of those n can be the deleted files of a service now. You can't make new local file services yet, obviously, but all search is now ready for it.

Since adding deleted files search was easy, I am adding it this week. Users in advanced mode will now see a list of deleted file domains on the tag autocomplete file domain button menu. While it is rare you ever want to do this, it was never actually possible to search these domains completely before, and definitely not quickly. Give it a go and let me know if you run into any trouble!

The next step here will be to write a new widget, probably some sort of checkbox list, that lets you select multiples of this new list. Then in future, if you had, say, a 'sfw' file service and a 'nsfw' service, you would be able to seach either or both at once easily. Then, I'll have a handful more things to do: an expansion to file import options to determine where imports are going, cleverer trash that supports n locations, and migration tools so you can move/copy between services, and then I _think_ I will be able to just let you add new local file services in manage services.

misc

I spent a little time with weird files this week. I added support for audio-only mkvs/webms and improved the test that checks whether a video with an audio track actually just has a silent one. I know I get annoyed when a video seems to have audio but actually doesn't, so this week will queue up all your videos for a recheck and hopefully fix a bunch. I also fixed the colours of some weird LAB TIFF files. If you have some jank test images in other unusual colourspaces, please send them in, it was fun figuring this one out.

The program should also shut down a little quicker now!
Open

R: 8 / I: 0 / P: 15

Version 468

https://youtube.com/watch?v=UGnbiBDmuBI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v468/Hydrus.Network.468.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v468/Hydrus.Network.468.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v468/Hydrus.Network.468.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v468/Hydrus.Network.468.-.Linux.-.Executable.tar.gz

I had a good first week back. A mix of bug fixes and little improvements.

couple of highlights, otherwise all misc this week

Thanks for the quick issue reports over the holiday. The scanbar had an annoying thing where the new single pixel border was making it awkward to drag when in borderless fullscreen--you'd move your mouse to the bottom of the screen, but then you'd click the border and not the scanbar, and the whole video would move--this I have now fixed. Also an issue with new imports' 'pixel duplicates' data not being saved correctly--I have fixed the problem and scheduled all affected files to regen their pixel duplicate data.

I also put some time into multiple local file services. It was more background work, this time to do with system predicate fetching, so there isn't much neat stuff to show off unless you are an advanced user. Almost all the background work is done now, though, so I hope to start spinning up more complex search UI and actually adding new file services pretty soon. One neat thing, now that I have new file filtering tools, I was able to expand 'system:file service' to search for deleted and petitioned files.

full list

- misc:
- fixed an issue where the one pixel border on the new 'control bar' on the media viewer was annoyingly catching mouse events at the bottom of the screen when in borderless fullscreen mode (and hence dragging the video, not scanning video position). the animation scanbar now owns its own border and processes mouse events on it properly
- fixed a typo bug in the new pixel hash system that meant new imports were not being added to the system correctly. on update, all files affected will be fixed of bad data and scheduled for a pixel hash regen. sorry for the trouble, and thank you for the quick reports here
- added a 'fixed font size example' qss file to the install. I have passed this file to others as an example of a quick way to make the font (and essentially ui scale) larger. it has some help comments inside and is now always available. the default example font size is lmao
- fixed another type checking problem for (mostly Arch/AUR) PyQt5 users (issue #1033)
- wrote a new display mappings maintenance routine for the database menu that repopulates the display mappings cache for missing rows. this task will be radically faster than a full regen for some problems, but it only solves those problems
- on boot, the program now explicitly checks if any of the database files are set as read-only and if so will dump out with an appropriate error
- rewrote my various 'file size problem' exception hierarchy to clearly split 'the file import options disallow this big gif' vs 'this file is zero size/this file is malformed'. we've had several problems here recently, but file import options rule-breaking should set 'ignore' again, and import objects should be better about ignore vs fail state from now on
- added more error handling for broken image files. some will report cleaner errors, some will now import
- the new parsing system that discards source urls if they share a domain with a primary import url is now stricter--now discarding only if they share the same url class. the domain check was messing up saving post urls when they were parsed from an api url (issue #1036)
- the network engine no longer sends a Range header if it is expecting to pull html/json, just files. this fixes fetching pages from nijie.info (and several other server engines, it seems), which has some unusual access rules regarding Range and Accept-Encoding
Open

R: 2 / I: 0 / P: 15

Version 467

https://youtube.com/watch?v=MrweXsImVhg [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v467/Hydrus.Network.467.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v467/Hydrus.Network.467.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v467/Hydrus.Network.467.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v467/Hydrus.Network.467.-.Linux.-.Executable.tar.gz

I had a good week cleaning some things up for end of year.

couple of scanbar things

I polished the new autohiding video scanbar. A bunch of the layout and coordinate detection code of the scanbar and the media canvas behind it is less jank. If you had some flicker or weird mouse-popup behaviour last week, I hope it is better now!

If you are a macOS user however, I held back one of the changes. There's a background-erasing hack I put in a couple years ago because without it macOS media viewers went to 100% CPU. I hope this is no longer true, but I'm not certain, so you still have the flag on by default. Please hit _help->debug->gui actions->macos anti-flicker test_ and then try browsing some images and video in a new viewer. Does your client lock up, or is it ok now? If everything is good, I'll remove the flag and you can get some nicer anti-flicker tech too.

full list

- new scanbar cleanup:
- the media container's scanbar and volume control are now combined on the same widget, meaning they now show/hide in sync and faster. their layout calculation is also more sensible. the new controls bar also has a thin border to make it pop better against a background video
- improved the way some auto-hide anti-flicker tech on the scanbar now works. it all hides a frame faster sometimes
- figured out some new anti-flicker tech to reduce/eliminate a frame of stretch when flicking from a static image to an mpv video, particularly for the first or second time in a session
- fixed a bug where clicking the global mute/unmute on an mpv player meant that certain shortcut keys (usually those with arrow keys) would not work on that player again. (it was a focus issue on the button, which then captured some form navigation keys but they had nowhere to go)
- brushed up some mouse coordinate testing logic across the program. some linux clients had trouble with the new animation scanbar popping up over mpv, I think I improved it!
- fixed another type problem with newer python/PyQt5 on Arch, also in scanbar coordinate testing
- fixed some dodgy colours in the scanbar initialisation and volume control border
- macOS users: I undid a long-time paint hack on the media container and the static image canvas. Qt is responsible for clearing the background again, which allows me to remove some jank anti-flicker tech. HOWEVER, the original reason for this hack was because without it, old macOS went to 100% CPU whenever the media viewer was showing something. therefore, to be safe, this option is still on for macOS users for now. you'll get a little flicker when browsing. please try hitting _help->debug->gui actions->macOS anti-flicker test_ and do some mixed video/image browsing. does your whole damn client lock up?

R: 0 / I: 0 / P: 15

Malformed Drive -- Help

I've got multiple drives I've tested Hydrus on, and can't figure out why I'm having issues. All my drives are healthy and run without problems. At first I thought Window's built in drive compression might be fucking with Hydrus, but I decompressed my whole SSD, and I'm still getting errors. Can someone suggest a possible reason for my issue? Will provide logs if needed, Thanks in advance.
Open

R: 4 / I: 0 / P: 15

Version 466

https://youtube.com/watch?v=kWXAL9bHciQ [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v466/Hydrus.Network.466.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v466/Hydrus.Network.466.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v466/Hydrus.Network.466.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v466/Hydrus.Network.466.-.Linux.-.Executable.tar.gz

I had a great week. The duplicate filter now supports filtering by pixel duplicates, and videos now fit better in the media viewer.

better duplicate search

Now we have pixel duplicate data stored in the database, I can search it. The duplicate filter page now has a dropdown that lets you select 'must be pixel dupes', 'can be pixel dupes', and 'must not be pixel dupes'. It does exactly what you think, and with luck it should be pretty fast no matter what you select.

Note that you may not have all your old pixel dupe data calculated yet. I started it for everyone last week, but even if it is doing 10,000 files a day, if you have a big client it will take a little longer. It may also still be working on the ICC profile queue. Hit up database->file maintenance->manage scheduled jobs to see how it is doing. You can rush it there if you like.

As I was working in this system, I discovered that at some point I had started recording at what search distance potential duplicates were matched. This is to say if you were set up to search at distance 4 and two files were matched as potential duplicates with a distance of 2, it would save that 2 into the database. I am not totally sure how retroactively accurate this data is, but I've added a control for it to the filtering panel too. You can now tell the filter to only present files that are 'exact match' at distance 0 and it should work mostly ok.

Some pretty complicated database work went into this. The most complicated search joins seven different things together. I know it is fast on my test machine, but if you have a really large client with a lot of searched dupes, some search types may be unbearably slow. Please let me know how you get on, and I'll optimise what I can.

scanbar

The audio/video scanbar in the media viewer is now embedded inside the media frame. It shows and autohides when your mouse moves closer and away. It now means you can now go borderless fullscreen on a 16:9 display and finally have a 16:9 video fit perfectly!

I have wanted to do this for a really long time, but some of the layout code here is really awkward, and getting widgets to pop on top of each other can be tricky. Thankfully it turns out Qt has a nice way to do it, so I've now hacked this together. There is still a small amount of jank--the scanbar and volume control are currently separate objects, so sometimes they'll show/hide in separate frames, and you might also see the scanbar nub pop in a frame late, but I can work on these issues in future. I will also add some options so you can change the size of the show/hide activation area around the scanbar. But for now, I am pretty happy with this.

If you are a keyboard user, please check out the new shortcut in the 'global' set that flips on/off a 'force the animation scanbar to show' mode. I don't really want to bring back the old always-on hanging-below scanbar, so I hope this will be a good enough substitute. But let me know, and if you really hate this new scanbar, we'll see what we can do.
Open

R: 3 / I: 0 / P: 15

Version 464

https://youtube.com/watch?v=mn34GKTVqMM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v464/Hydrus.Network.464.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v464/Hydrus.Network.464.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v464/Hydrus.Network.464.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v464/Hydrus.Network.464.-.Linux.-.Executable.tar.gz

I had a good week. Some images get better colour, and the software now deletes files more neatly.

ICC profiles

tl;dr: Some image colours are better now.

Some images come with an 'ICC profile'. A scan of one ICC-heavy client showed that 10% of all images had them. ICCs are basically colour correction data like 'actually, the blue should be this much more intensely blue than is stored' in a variety of sometimes complicated ways. Some nice cameras attach a fast ICC based on lighting conditions or known defects in the camera, and some image editing software does similar. ICCs are also used for broader and more complicated colour conversion tasks like in the migration from sRGB to HDR formats if you have a new tv/monitor.

Hydrus now has basic ICC tech. It recognises when an image has an ICC and will apply it, converting the rendered colour to 'normal'. This happens right on image load, so it should apply in every media viewer and all new thumbnails. Should work for the server, too. With luck, this will be one of those things where you don't really notice any difference, but you should just have some nicer pictures, mostly photos, in your client. Let me know if you have any files that you know are still not rendering right!

This work took a long time to get done, but a user helped me out with some great examples and more background info on how to handle it all properly. The next step will be to add more indication of when an image has an ICC and add the ability to flip it on and off. Another topic is 'display ICCs', where some artists and other enthusiasts will have an ICC calibrated for their particular monitor or for different display conditions (e.g. 'how would this look like under this lighting?' or '...from this printer?'). Since we have the tech now, I think I should be able to support this sort of ICC conversion too.
Open

R: 3 / I: 0 / P: 15

Version 463

https://youtube.com/watch?v=Xm3mrwyR2pw [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v463/Hydrus.Network.463.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v463/Hydrus.Network.463.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v463/Hydrus.Network.463.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v463/Hydrus.Network.463.-.Linux.-.Executable.tar.gz

I had a great week. I was able to catch up on the work I wanted and do some neat stuff besides.

import presentation

I completely overhauled the importer 'presentation' options, which you can see under all 'file import options'. These have always been a bit advanced and confusing, so I rewrote it to specifically say a more human-friendly summary like 'show new files' and added text and tooltips to better explain what was going on. It also works a bit faster and cleaner.

You can also now say 'show inbox files', regardless of 'successful' or 'already in db', which I think may be neat as the default for a watcher page that you process and revisit several times--every time you were to re-highlight a watcher, it would filter out anything you previously archived.

And I updated the right-click menu on downloader and watcher pages' lists, where it said "show importers' new files", to use the new options. It now tucks it into a submenu, adds the 'inbox only' option, and if only one importer is selected it says what the default actually is and removes any duplicate action.

other highlights

The client now recognises ogg files with video as ogv! Previously, they were always considered audio, but now it will make thumbnails and parse resolution and duration and let you zoom properly with the video player and so on. All existing ogg files will be scheduled for a rescan when you update, so your ogvs should just pop into place.

I think I fixed the deviant art tag search downloader. They changed their tag search routine recently, so I had to change some things behind the scenes. Everyone will get it on update, and the existing 'deviant art tag search' downloader should just work again.

The manage ratings dialog now has copy/paste buttons for quick copying of multiple ratings across sets of files. This is a prototype, so give it a go and let me know what you think. It copies 'null' ratings at the moment, but maybe we'll want a second option too that only copies set ratings.
Open

R: 2 / I: 0 / P: 15

Version 462

https://youtube.com/watch?v=Gu2Iayi3AvM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v462/Hydrus.Network.462.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v462/Hydrus.Network.462.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v462/Hydrus.Network.462.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v462/Hydrus.Network.462.-.Linux.-.Executable.tar.gz

I had a good week with a variety of fixes and optimisations. New users also get a couple of user-friendly services to start with.

highlights

A recent update to the mpv seekbar accidentally caused bad errors during audio playback. I have now fixed it. I regret the regression, and I have added tests to make sure this does not happen again.

All 'file log' import queues now search and count themselves faster. Any heavy import session, like a very large downloader or many medium size downloaders working at once, should use less CPU just to do normal work.

I sped up some file searches that were running super slow. The general problem file search was typically a mix of 'has multiple search predicates' and 'includes a short unnamespaced tag or namespace:anything'. This combination was taking two minutes or more rather than a handful of seconds due to an overly complicated database query that wasn't running well. Should be fixed now, but let me know if you have any more of this sort of trouble!

new user services

To help new users, new clients now start with two extra services:

First is a local tag service called 'downloader tags', which the initial default tag import options is now set to get tags for. I hope this will reduce some of the confusion around default tag import options, merged downloader/personal tags in 'my tags', and having to retroactively fill in missed tags for files downloaded early on.

Second is a like/dislike rating service called 'favourites'. Many users are unaware of the existence of ratings services at all, so now everyone will start with a simple clickable star. When you update, if you currently have no rating services, you will be asked if you want to get this new default. You don't have to, but if you haven't tried ratings out yet, give it a go!

full list

- misc:
- fixed a recent serious regression that could cause a crash when playing audio in mpv (issue #1007)
- the main importer file log now does 'get next/all/count imports with status y' calls significantly faster, particularly on very large lists. these calls happen all the time for different status text changes and general 'which import to try next?' work. all busy downloader situations should see CPU gains to regular and background work
- fixed a problem where importing with the min/max file resolution options set would give a typo error when the size was violated rather than 'ignored'
- I think I have fixed an issue with subscriptions not wanting to run a query if by random accident that query has an invalid URL selected as the query's 'example url' for various pre-work login and bandwidth tests
- hydrus can now capture duration/fps of videos that specify two very close fps, e.g. 60 and 59.99. previously, these would default to the 24 fallback and could cause some weirdness in mpv
- replaced the default pixiv artist page api parser with one that fetches the newer url format, matching the tag search. existing users will see no automatic change but will receive the new parser, so if you are a big pixiv user, you might like to switch 'pixiv artist gallery page api' to the 'new urls' parser variant under _network->downloader components->manage url class links_. note that if you do this, your pixiv artist subscriptions will do a mini-resync (which involves some wasted time/bandwidth) as their urls change!
Open

R: 3 / I: 0 / P: 15

Version 461

https://youtube.com/watch?v=oiP5bKlJ0fM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v461/Hydrus.Network.461.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v461/Hydrus.Network.461.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v461/Hydrus.Network.461.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v461/Hydrus.Network.461.-.Linux.-.Executable.tar.gz

I had a great week doing a lot of background cleanup work, and I also fixed some things and improved some quality of life.

highlights

I brushed up last week's .clip file metadata parsing. Some clip files were getting too-small resolutions and thumbnails due to a 'canvas unit' issue, basically some clips store resolution in mm or inches etc... rather than pixels. Thanks to a user who also provided some great examples, we managed to pin down a solution to get actual pixel resolution, and I also figured out duration and number of frames for animated clips. All clips will reparse again on update and generate nicer size thumbs if they need it.

Also, if you ever got a webm that seemed normal except it somehow had a crazy 7 hour duration, I think I figured out the problem here too. All superlong-but-actually-small files will be reparsed on update and should get a more accurate duration.

When you right-click on tags in the taglist, the 'search' submenu now provides actions to require/exclude 'namespace:anything' if your selection all shares the same namespace.

Also, as a little test, I am trying out a new quiet shortcut on the file selection taglist. If you now hold down control when double-clicking or hitting enter on tag(s) here, they now go up to the search negated. Double-click = 'add tag to search', ctrl+double-click = 'add -tag to search'. Of course ctrl+click is awkward since it can cause a deselection, so I think this could do with some more work, and eventual integration into the shortcuts system so you can change it as you like, but let me know what you think. A user recommended adding the +/- buttons as you see on some boorus, so this is my interim step towards that.

When you paste query texts into the edit subscription dialog, if any of the texts you paste are already in the subscription and currently DEAD, the dialog now revives those subs (basically the same as 'check now'). I have personally wanted this for ages and kept forgetting to add it. The sub will just do another quick check on what you paste, just in case that query started getting things again recently, rather than swallowing your input as forevermore DEAD.

I moved some autocomplete options from 'gui pages' options panel to 'search'. If you are a Linux user who has trouble with the floating autocomplete dropdown window, check it out and see if 'embedding' the dropdown works better for you.
Open

R: 2 / I: 0 / P: 15

Version 460

https://youtube.com/watch?v=gP5LXd6DFx4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v460/Hydrus.Network.460.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v460/Hydrus.Network.460.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v460/Hydrus.Network.460.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v460/Hydrus.Network.460.-.Linux.-.Executable.tar.gz

I had an ok but light week of various work.

misc

When I improved the 'apng' file parsing the other week, there was a mistake in how I hooked it up and not all the file metadata was being propagated to the database. A couple of users got some weird '1 frame' apngs as a result. I have fixed this problem, and improved the parsing to deal with more sorts of apngs, and scheduled a complete rescan of all your apngs to get some nicer 'number of frames' counts.

In a related thing, a user sent in a way to get some cool metadata from 'clip' files. These now get resolution and a rough thumbnail. I have scheduled all your clip files, if you have any, to update too.

The Client API now supports a better way to refer to tag services when editing tags. If you are a Client API developer, please check out the updated help--you can now user service keys instead of service names. I am in no rush to delete the 'names' parameters, but I expect I will eventually. 'names' was always an early hack that has flaws as an identifier, so when it is convenient, please try to migrate to /get_services and service keys.

A new help file in install_dir/db directory, 'help my mpv crashes with WASAPI or ASIO audio', now has a neat couple lines to add to your mpv.conf if you have this problem. Many thanks to the user who figured this out!

full list

- client api:
- /add_tags/add_tags now supports 'service_keys_to_tags' and 'service_keys_to_actions_to_tags'.
- /add_urls/add_url now supports 'service_keys_to_additional_tags'
- /get_files/file_metadata now returns with duplicates of the tag structures using 'service_keys_to_statuses_to_(display_)tags'
- added unit tests for the above
- updated the client api help for the above
- I recommend you move from 'service_names_to...' to 'service_keys_to...' when convenient. 'names' was an ugly old hack, and while I am not in a rush to delete it from the client api, I think I will eventually
- client api version is now 21
- .
- apngs and clips:
- fixed a problem where the new apng metadata parsing was not completely hooked up, so num_frames was not being parsed correct for the final metadata row even when 'apng' filetype was, leading to some odd '1 frame apng' situations
- apng parsing now recognises more kinds of apngs--if one has an unusual scaling chunk in the header, this is recognised and the correct animation chunk searched for
- clip files now get resolution on import and a thumbnail! big thanks to the user who dug up how to extract this--it actually comes from a sqlite file embedded in the clip! (issue #996)
- on update, all apngs and clips will be scheduled for metadata rescan, and all clips will be scheduled for thumbnail generation
Open

R: 3 / I: 0 / P: 16

Version 459

https://youtube.com/watch?v=gZgbF2TRK2k [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v459.2/Hydrus.Network.459.2.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v459.2/Hydrus.Network.459.2.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v459.2/Hydrus.Network.459.2.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v459.2/Hydrus.Network.459.2.-.Linux.-.Executable.tar.gz

Hey, the release I put out earlier had a problem! The links above point to the fixed version. If you got the old one within an hour of me putting this post up, please download and install/extract again and you'll be able to boot. Sorry for the trouble!

I had a great week mostly clearing out bugs.

highlights

I gave the new duplicate filter zoom locking calculation another pass. Thank you to the users who submitted examples of my last attempt not working right. It is now more careful and targeted towards fixing the 'a watermark banner spilled over my monitor edge' problem, and it deals with different combinations of portrait and landscape images and viewers better. Let me know if you still have any borked zooms!

The client's downloader system can now deal with some 'partial/interrupted download' situations and automatically resume from where it left off. If you ever saw a network job widget error out with 'was expecting 21.3MB but got 458KB', this will try to fix that!

The program should be better about saving the work of subscriptions that are running when you exit the program. The way their jobs ran was a little hacky, previously, so I moved them to the main worker pool everything else runs on and brushed up their shutdown signals. Let me know if you encounter any situations of them not shutting down nice and fast with client shutdown.

To help with debugging and other screenshotted reports, the UI now puts the program version number in the title of windows and dialogs. Also, if you previously set a custom 'main gui title' under options->gui, I have reset this this week and renamed it to 'application display name'--it now shows on all windows.

Big Sur macoS clients have had a problem for a while where menus spawned from dialogs can't be clicked. It seems to be a weird Qt bug. Until there is a better fix, I hacked together a new window to display the menu items instead. If you are a Big Sur user, let me know how it works for you. You can turn it off if you need to under options->gui.

I wrote a new help document, 'help my media files are broke.txt', in the install_dir/db directory. This collects the different repair routines I have developed with users who have run into trouble with broken drives or incomplete backup restores. If you are missing many files--or you know you have many surplus files--and were not sure how to fix the situation, please check it out!

I fixed the new 'advanced file deletion' checkboxes under options->files and trash! Sorry, they weren't hooked up right in 458!
Open

R: 4 / I: 0 / P: 16

Version 458

https://youtube.com/watch?v=45pF6YtMvQ4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v458/Hydrus.Network.458.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v458/Hydrus.Network.458.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v458/Hydrus.Network.458.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v458/Hydrus.Network.458.-.Linux.-.Executable.tar.gz

I had a good week. I fixed some more bugs and improved some quality of life.

highlights

I fixed an html parsing problem that started in the past few weeks in the release build. I understand it broke Newgrounds searching, but it should be fixed now, so if this affected you please give it another go.

You can now set the client to always boot with network traffic paused under network->pause. Useful if you have an unusual network/vpn situation that is often not connected!

I have improved the duplicate filter zoom lock calculation. Previously, it forced the width of the two images to be the same--now it chooses height if it thinks that fits things better. This should stop 'B' images with a watermark banner on the bottom accidentally spilling that watermark difference below the screen.

advanced user highlights

I improved some repository update error handling and auto-healing. If you have had an issue processing the PTR related to 'this file was the wrong type' that did not seem to fix by itself, please unpause processing and let it try one more time.

If you use the advanced file deletion dialog, you can now choose to remember the last used reason and action under options->files and trash. I also fixed a bug where some number of files were forgetting their given deletion reason once they were permanently deleted from the trash. I am sorry for the trouble with this bug, and I will make sure to add unit tests so it cannot happen again.

File import objects now track 'primary' URLs (that directly lead to a file download) separately from parsed 'source' URLs. They are now split under the new 'file log' URLs menu, which also now shows the referral URL. Furthermore, file import options now lets you choose whether primary URLs are associated with imports (which you might want to turn off if you are doing some clever one-time proxy download etc...).
Open

R: 3 / I: 0 / P: 16

Version 457

https://youtube.com/watch?v=T4D3Qmn0nuY [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v457/Hydrus.Network.457.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v457/Hydrus.Network.457.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v457/Hydrus.Network.457.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v457/Hydrus.Network.457.-.Linux.-.Executable.tar.gz

I had a good week. Some UI jank should be much better!

highlights

I reworked how the menubar menus (file, network, pending etc...) update this week. They update labels and swap items in and out more efficiently. Most importantly, the 'pending' menu should now update with no flicker at all, so if you have a bunch of downloaders/subscriptions running and continually pending new tags, you should be able to commit and upload with no trouble! Let me know how it works for you.

I also removed some ancient and now obsolete safety checks that stop the client from updating certain UI while the program is minimised. With luck, when you restore the client--and more significantly, when you restore it from minimised to system tray--there should be less judder, and hopefully no freezing, since the client no longer has to catch up with hundreds or thousands of updates. If you minimise to system tray and have had hangs, let me know if the situation is better now. Also, if you suddenly get popup message sizing problems or similar UI jank after minimising or putting the client on a virtual desktop or similar, please let me know.

The client now uses much less CPU to parse and import png files!

I updated the mid-tier file searching help here: https://hydrusnetwork.github.io/hydrus/help/getting_started_more_files.html
Open

R: 2 / I: 0 / P: 16

Version 456

https://youtube.com/watch?v=MxaD8_Vazpo [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v456/Hydrus.Network.456.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v456/Hydrus.Network.456.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v456/Hydrus.Network.456.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v456/Hydrus.Network.456.-.Linux.-.Executable.tar.gz

I had an ok week. With luck, the client should have less UI lag, and I also fixed a bunch of stuff and improved some quality of life.

basic highlights

I removed some very hacky memory management code this week. It was eating far more CPU than it was worth, particularly for large clients. If you have a very heavy client, particularly if it has lots of heavy subscriptions, please let me know if you A) have fewer UI lockups, and B) see any crazy memory spikes while running subs. As bad as the old routine was, it was aggressive and effective at what it did, so I may have to revisit this.

All multi-column lists across the program now catch enter/return keystrokes and trigger an 'activate' call, as if you had double-clicked. Should be easy to navigate and highlight a downloader page list just with the keyboard now.

advanced highlights

The 'file log' window now lets you see and copy an import item's hashes, tags, and urls from its right-click menu. I hope this will help debug some weirder downloader problems and generally inform on how the downloaders work.

The Edit URL Class dialog has had a layout makeover. Also, URL Classes now support matching and normalising 'single value' parameters (this is where you have a token/keyword parameter rather than the traditional key=value pair).

The hydrus server now remembers custom update and anonymisation periods! Previously, it was resetting to defaults on a restart! Thank you for the reports here--I apologise for the inconvenience and delay.

I added an 'mpv report mode' to the debug menu. If mpv loads for you but you have silent audio or similar (and perhaps some crashes, but we'll see if this catches useful info in time), this'll dump a huge amount of mpv debug information to the log.
Open

R: 5 / I: 1 / P: 16

Version 455

https://youtube.com/watch?v=Gs069dndIYk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v455/Hydrus.Network.455.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v455/Hydrus.Network.455.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v455/Hydrus.Network.455.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v455/Hydrus.Network.455.-.Linux.-.Executable.tar.gz

I had an ok couple of weeks. Some quality of life is improved, the UI should save sessions a bit quicker, and the database repairs itself more of itself.

Your client may warn you about a missing index on first boot. If it does, that's great--some new repair code is working!

session save

Some users with large sessions (typically around 2-3+ million 'weight') are experiencing severe UI lockups and general lag. This week I deal with part of that problem by greatly reducing the amount of CPU needed to save most sessions. Now pages know if they have been changed since being loaded, and if they have no changes, they can skip a bunch of session save work.

There's nothing you have to do here, but if you have been experiencing 12+ second freezes every few minutes, please let me know how you get on now. I can't promise things will be perfect, but if you have any large and inactive downloader pages, you should notice things running a bit smoother (exiting the program too, should be faster).

There is still a problem of lag and lockups when there are 200+ downloaders in the session. This is a different and unfortunately more severe problem that I have relieved in recent weeks but needs some other large work to completely fix. Best solution for now is to clear out finished downloaders if you can.

misc

Many of the simple system predicates, like height and filesize, now support the '≠' (not equal) operator! This is still in the early stages, but I'd like to add some options of the thumbnail menu so you can say 'find files that (do not) have this file's precise resolution/duration/size' etc...

You can now copy thread subjects to clipboard in the watcher downloader page list!

The advanced file deletion dialog now remembers the last reason you chose!

The network job widget (which shows download progress, you see it on any downloader) now shows the current URL it is working on under its cog menu. You can click it to copy to clipboard. May help to debug some odd network errors etc...

The way data sizes over 1KB are rendered has been overhauled. It should now generally be to three significant figures, so 3.11MB instead of 3.1MB, 12.3GB instead of 12GB. See if you like the change, and if not, try tweaking it to 2sf under the new EXPERIMENTAL option in options->gui.
Open

R: 3 / I: 0 / P: 16

Version 454

https://youtube.com/watch?v=Ch24PpzMKog [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v454/Hydrus.Network.454.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v454/Hydrus.Network.454.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v454/Hydrus.Network.454.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v454/Hydrus.Network.454.-.Linux.-.Executable.tar.gz

I had an ok week mostly working on making images look better.

warped tiles

tl;dr: Images look better now.

I put a lot of time into the tiled renderer this week, mostly fixing 'stretch-warped' tiles. These are most obvious in the duplicate viewer when you are flipping between two very similar images that have slightly different resolution--you would see one part of one of the images stretched one pixel horizontally or vertically compared to the other.

This turned out to be complicated to fix, and moreso for unusual zooms (like a 124.65% style canvas fit zoom), but I think I have fixed almost all warping, and tile join artifacts, for all zooms below about ~6000%, with the exception of some bottommost and rightmost tiles.

I also added an option to options->speed and memory to govern how large the tiles should be. This is 768-pixel squares by default, but if you still have trouble with warping in the duplicate viewer or other high zoom situations, please try boosting it a bit. Larger numbers will be less memory and CPU efficient, so I don't recommend 4096 to start off, but let's see what we find in real situations.

I like the tiled renderer, so I don't want to abandon or completely replace it. I thought seriously of ditching it this week, when I could not figure out a solution to warping, but I slept on it and figured it out. If I can keep patching remaining problems, I'll be happy, but I do have a backup plan as well. Let me know how you get on!

the rest

I think I fixed the issue where, after some serious network errors, the downloaders could get in a 'traffic jam' and all stay on 'pending', with nothing going to 'working'.

Finishing a big archive/delete filter should result in less UI lag. If you are a user who hits F5 right after committing archive/delete, let me know how this works for you!

The green/blue/red texts in the duplicate filter's right-hand hover window are now moved to the style system, so QSS stylesheet files (as under options->style) can finally change their colour. I set up some basic dark/light blues for all the default QSS files, so it should be a bit easier to read the blue text in dark mode styles now. These were the last hardcoded text colours in the program, so we are now one step closer to moving everything to QSS.
Open

R: 3 / I: 0 / P: 16

Version 453

https://youtube.com/watch?v=YA6A8xohpxc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v453/Hydrus.Network.453.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v453/Hydrus.Network.453.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v453/Hydrus.Network.453.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v453/Hydrus.Network.453.-.Linux.-.Executable.tar.gz

I had a great week. I ended up mostly fixing bugs. If you have a large client, the update may take a few minutes this week.

bug fixes

I discovered a semi-important tag processing bug last week--for some users, files that were imported before they added the PTR were not getting all the correct tag info and would not appear in some tag searches. I fixed the underlying file-tracking bug and have added a retroactive 'gap-filler' to this week's update. If you used the client for a long time but added/reset the PTR recently, you'll have more gaps so your update will take longer.

Another persistent issue has been the 'tiling artifacts' with my new tile-based image renderer, where at certain zooms an image would have one or more horizontal or vertical blurred/janked line of pixels. You'd notice it on cleaner vector images where a smooth curved black line would suddenly jag at one zoom. I had trouble reproducing this at first, but some users got it quite often, and with their help I was able to figure it out. I have rewritten the part of my renderer that was failing, and I think I now have the artifacts completely fixed for all 'normal' zooms. Let me know how you get on!

misc

Network job widgets have a little user-friendly update this week. I brushed up some of the status texts, made the text lay out better, and when the widget counts down waiting for something (like free bandwidth), if it doesn't get it because another downloader did, it says so before resetting the countdown clock.

A helpful user wrote a new darkmode style called OledBlack. Check it out under options->style!

I added OR predicates to Client API file search!

If you are an advanced user, you might like to read this idea I had for setting up multi-tag processing workflows: https://hydrusnetwork.github.io/hydrus/help/advanced_parents.html#parent_favourites I suddenly thought of it a couple weeks ago and it worked so well for me that I decided to write it up. It basically involves using local parent tags to group PTR tags into a pseudo-OR search that you can easily edit.
Open

R: 2 / I: 0 / P: 16

Version 452

https://youtube.com/watch?v=iePWOFOSl_U [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v452/Hydrus.Network.452.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v452/Hydrus.Network.452.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v452/Hydrus.Network.452.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v452/Hydrus.Network.452.-.Linux.-.Executable.tar.gz

I had an ok week. I ended up doing a lot of boring behind the scenes prep, but there is also some nice new quality of life. If you sync with the PTR, update will take a few minutes this week.

all misc this week

After a very long delay, I finally have shortcuts for seeking video back/forwards in the media viewer. New users will get ctrl+left/right to seek back 2.5 seconds or forwards 5 seconds. Existing users need to add their own--please check it under the 'media viewers - all' shortcut set. You can set whatever seek distance you like, and even set multiple with different distance jumps. Sorry for how long it took to get this in--I had to update my shortcut system first!

I did some behind the scenes tag work this week. The database can now handle larger 'tag as number' searches. Update will take a few minutes to load these bigger numbers into the fast search cache.

If you have a subscription that is not meant to completely sync (e.g. it pulls a sample from a gallery with unusual sort order, or you have the 'normal file limit' intentionally small), there is a new checkbox that suppresses the 'hey, there was a gap, click here to fill it in' popup windows.

full list

- misc:
- my 'simple' shortcut commands can now store additional variables. to start things off, I have finally added 'seek video' shortcut commands that have back/forwards and second+millisecond values. these should work on the native video viewer and mpv, audio and video. existing users will have to add their own (do it to the 'media viewers - all' set), but new users will get ctrl+left/right for 2.5s back and 5s forwards as the new defaults. let me know if you have any trouble!
- the maximum number tracked by 'tag as number' system predicate is expanded from -99999999->99999999 to -2^63 -> 2^63 - 1. tag caches will be regenerated on update to store these, it will take a few minutes. the input ui for the system predicate is temporarily limited to -/+2^31, but I'll expand it
- subscriptions now have a checkbox for 'do not worry about subscription gaps'. if you have a subscription that gets files randomly, or gets an intentionally small sample, this will disable the 'hey, there was a gap, click here to fill it in' popup messages
- you can now set negative values for the duplicate score weights in options->duplicates
- also added a weight for the 'nice ratio' duplicate comparison

R: 69 / I: 13 / P: 16

Bug Reports

Please feel free to submit any bug reports here.
Open

R: 3 / I: 0 / P: 16

Version 450

https://youtube.com/watch?v=fLLKwIfs1NM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v450/Hydrus.Network.450.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v450/Hydrus.Network.450.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v450/Hydrus.Network.450.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v450/Hydrus.Network.450.-.Linux.-.Executable.tar.gz

🎉🎉 MERRY 450! 🎉🎉

I had an ok week. Last week's experimental release went well, so I have polished that, and I otherwise caught up with a variety of small work.

If you sync with the PTR, update will take a couple of minutes.

all misc this week

So, the update storage change went ok! There were a couple of little sync bugs to clear up, but overall it works--tag repositories now track their processing progress by mappings, siblings and parents separately. You don't have to do anything, and this doesn't matter much for day to day work, but it allows for individualised reset and reprocessing. All 448 users will have their siblings and parents reset and reprocessed, which will take a couple of minutes to do on update, and about fifteen minutes on your next processing job to fill back in, and which should eliminate some bad siblings and parents due to years-old processing bugs that long term users have been dealing with (leaving only current bugs, which I am also working on). The reset will not delete any pending siblings or parents you have, so no worries if you have a bunch waiting to be uploaded.

Advanced Users: The PTR sibling and parent reset will however remove any siblings and parents you uploaded that were then denied by jannies (which your client would have added to itself anyway). Everyone is reset to a 'clean' sync with this change, so if you know you have a ton of surplus denied siblings you rely on, perhaps from years ago that we agreed I would deny on the PTR to help you hack in an overwrite in the old system, you might like to hold off updating and first figure out a PTR sibling/parent backup to a local service using tag migration.

I fixed some things with Mr Bones. His numbers are accurate to your 'my files' again, and he now talks about your total deleted files and also your earliest file import time. I divided the ugly growing stack of numbers into tabs, which I am sort of happy with, sort of not. People like to take screens of Mr Bones, but they have different preferences on what to show, so I may replace this with expand/collapse buttons or similar, so you can show everything if you want.

If you use the export files window to get a lot of files out, it now makes a progress popup. You can close the window while it is exporting and still see and cancel the job.

The Client API file search now supports file and tag domain selection (like the 'my files' and 'all known tags' buttons on a normal search page), and also file sort for searches. I know the Client API guys have been waiting on this, so with luck we should see some neat new search options in the Client API programs in the near future.

Top