Catalog of /hydrus/

Bottom

Mode: Thread

Max file size: 350.00 MB

Max files: 5

Remember to follow the rules

Max message length: 4096

Open

R: 3 / I: 0 / P: 1

Version 458

https://youtube.com/watch?v=45pF6YtMvQ4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v458/Hydrus.Network.458.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v458/Hydrus.Network.458.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v458/Hydrus.Network.458.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v458/Hydrus.Network.458.-.Linux.-.Executable.tar.gz

I had a good week. I fixed some more bugs and improved some quality of life.

highlights

I fixed an html parsing problem that started in the past few weeks in the release build. I understand it broke Newgrounds searching, but it should be fixed now, so if this affected you please give it another go.

You can now set the client to always boot with network traffic paused under network->pause. Useful if you have an unusual network/vpn situation that is often not connected!

I have improved the duplicate filter zoom lock calculation. Previously, it forced the width of the two images to be the same--now it chooses height if it thinks that fits things better. This should stop 'B' images with a watermark banner on the bottom accidentally spilling that watermark difference below the screen.

advanced user highlights

I improved some repository update error handling and auto-healing. If you have had an issue processing the PTR related to 'this file was the wrong type' that did not seem to fix by itself, please unpause processing and let it try one more time.

If you use the advanced file deletion dialog, you can now choose to remember the last used reason and action under options->files and trash. I also fixed a bug where some number of files were forgetting their given deletion reason once they were permanently deleted from the trash. I am sorry for the trouble with this bug, and I will make sure to add unit tests so it cannot happen again.

File import objects now track 'primary' URLs (that directly lead to a file download) separately from parsed 'source' URLs. They are now split under the new 'file log' URLs menu, which also now shows the referral URL. Furthermore, file import options now lets you choose whether primary URLs are associated with imports (which you might want to turn off if you are doing some clever one-time proxy download etc...).
Open

R: 3 / I: 0 / P: 1

Version 457

https://youtube.com/watch?v=T4D3Qmn0nuY [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v457/Hydrus.Network.457.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v457/Hydrus.Network.457.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v457/Hydrus.Network.457.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v457/Hydrus.Network.457.-.Linux.-.Executable.tar.gz

I had a good week. Some UI jank should be much better!

highlights

I reworked how the menubar menus (file, network, pending etc...) update this week. They update labels and swap items in and out more efficiently. Most importantly, the 'pending' menu should now update with no flicker at all, so if you have a bunch of downloaders/subscriptions running and continually pending new tags, you should be able to commit and upload with no trouble! Let me know how it works for you.

I also removed some ancient and now obsolete safety checks that stop the client from updating certain UI while the program is minimised. With luck, when you restore the client--and more significantly, when you restore it from minimised to system tray--there should be less judder, and hopefully no freezing, since the client no longer has to catch up with hundreds or thousands of updates. If you minimise to system tray and have had hangs, let me know if the situation is better now. Also, if you suddenly get popup message sizing problems or similar UI jank after minimising or putting the client on a virtual desktop or similar, please let me know.

The client now uses much less CPU to parse and import png files!

I updated the mid-tier file searching help here: https://hydrusnetwork.github.io/hydrus/help/getting_started_more_files.html
Open

R: 2 / I: 0 / P: 1

Version 456

https://youtube.com/watch?v=MxaD8_Vazpo [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v456/Hydrus.Network.456.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v456/Hydrus.Network.456.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v456/Hydrus.Network.456.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v456/Hydrus.Network.456.-.Linux.-.Executable.tar.gz

I had an ok week. With luck, the client should have less UI lag, and I also fixed a bunch of stuff and improved some quality of life.

basic highlights

I removed some very hacky memory management code this week. It was eating far more CPU than it was worth, particularly for large clients. If you have a very heavy client, particularly if it has lots of heavy subscriptions, please let me know if you A) have fewer UI lockups, and B) see any crazy memory spikes while running subs. As bad as the old routine was, it was aggressive and effective at what it did, so I may have to revisit this.

All multi-column lists across the program now catch enter/return keystrokes and trigger an 'activate' call, as if you had double-clicked. Should be easy to navigate and highlight a downloader page list just with the keyboard now.

advanced highlights

The 'file log' window now lets you see and copy an import item's hashes, tags, and urls from its right-click menu. I hope this will help debug some weirder downloader problems and generally inform on how the downloaders work.

The Edit URL Class dialog has had a layout makeover. Also, URL Classes now support matching and normalising 'single value' parameters (this is where you have a token/keyword parameter rather than the traditional key=value pair).

The hydrus server now remembers custom update and anonymisation periods! Previously, it was resetting to defaults on a restart! Thank you for the reports here--I apologise for the inconvenience and delay.

I added an 'mpv report mode' to the debug menu. If mpv loads for you but you have silent audio or similar (and perhaps some crashes, but we'll see if this catches useful info in time), this'll dump a huge amount of mpv debug information to the log.
Open

R: 5 / I: 1 / P: 1

Version 455

https://youtube.com/watch?v=Gs069dndIYk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v455/Hydrus.Network.455.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v455/Hydrus.Network.455.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v455/Hydrus.Network.455.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v455/Hydrus.Network.455.-.Linux.-.Executable.tar.gz

I had an ok couple of weeks. Some quality of life is improved, the UI should save sessions a bit quicker, and the database repairs itself more of itself.

Your client may warn you about a missing index on first boot. If it does, that's great--some new repair code is working!

session save

Some users with large sessions (typically around 2-3+ million 'weight') are experiencing severe UI lockups and general lag. This week I deal with part of that problem by greatly reducing the amount of CPU needed to save most sessions. Now pages know if they have been changed since being loaded, and if they have no changes, they can skip a bunch of session save work.

There's nothing you have to do here, but if you have been experiencing 12+ second freezes every few minutes, please let me know how you get on now. I can't promise things will be perfect, but if you have any large and inactive downloader pages, you should notice things running a bit smoother (exiting the program too, should be faster).

There is still a problem of lag and lockups when there are 200+ downloaders in the session. This is a different and unfortunately more severe problem that I have relieved in recent weeks but needs some other large work to completely fix. Best solution for now is to clear out finished downloaders if you can.

misc

Many of the simple system predicates, like height and filesize, now support the 'โ‰ ' (not equal) operator! This is still in the early stages, but I'd like to add some options of the thumbnail menu so you can say 'find files that (do not) have this file's precise resolution/duration/size' etc...

You can now copy thread subjects to clipboard in the watcher downloader page list!

The advanced file deletion dialog now remembers the last reason you chose!

The network job widget (which shows download progress, you see it on any downloader) now shows the current URL it is working on under its cog menu. You can click it to copy to clipboard. May help to debug some odd network errors etc...

The way data sizes over 1KB are rendered has been overhauled. It should now generally be to three significant figures, so 3.11MB instead of 3.1MB, 12.3GB instead of 12GB. See if you like the change, and if not, try tweaking it to 2sf under the new EXPERIMENTAL option in options->gui.
Open

R: 3 / I: 0 / P: 1

Version 454

https://youtube.com/watch?v=Ch24PpzMKog [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v454/Hydrus.Network.454.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v454/Hydrus.Network.454.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v454/Hydrus.Network.454.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v454/Hydrus.Network.454.-.Linux.-.Executable.tar.gz

I had an ok week mostly working on making images look better.

warped tiles

tl;dr: Images look better now.

I put a lot of time into the tiled renderer this week, mostly fixing 'stretch-warped' tiles. These are most obvious in the duplicate viewer when you are flipping between two very similar images that have slightly different resolution--you would see one part of one of the images stretched one pixel horizontally or vertically compared to the other.

This turned out to be complicated to fix, and moreso for unusual zooms (like a 124.65% style canvas fit zoom), but I think I have fixed almost all warping, and tile join artifacts, for all zooms below about ~6000%, with the exception of some bottommost and rightmost tiles.

I also added an option to options->speed and memory to govern how large the tiles should be. This is 768-pixel squares by default, but if you still have trouble with warping in the duplicate viewer or other high zoom situations, please try boosting it a bit. Larger numbers will be less memory and CPU efficient, so I don't recommend 4096 to start off, but let's see what we find in real situations.

I like the tiled renderer, so I don't want to abandon or completely replace it. I thought seriously of ditching it this week, when I could not figure out a solution to warping, but I slept on it and figured it out. If I can keep patching remaining problems, I'll be happy, but I do have a backup plan as well. Let me know how you get on!

the rest

I think I fixed the issue where, after some serious network errors, the downloaders could get in a 'traffic jam' and all stay on 'pending', with nothing going to 'working'.

Finishing a big archive/delete filter should result in less UI lag. If you are a user who hits F5 right after committing archive/delete, let me know how this works for you!

The green/blue/red texts in the duplicate filter's right-hand hover window are now moved to the style system, so QSS stylesheet files (as under options->style) can finally change their colour. I set up some basic dark/light blues for all the default QSS files, so it should be a bit easier to read the blue text in dark mode styles now. These were the last hardcoded text colours in the program, so we are now one step closer to moving everything to QSS.
Open

R: 3 / I: 0 / P: 1

Version 453

https://youtube.com/watch?v=YA6A8xohpxc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v453/Hydrus.Network.453.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v453/Hydrus.Network.453.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v453/Hydrus.Network.453.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v453/Hydrus.Network.453.-.Linux.-.Executable.tar.gz

I had a great week. I ended up mostly fixing bugs. If you have a large client, the update may take a few minutes this week.

bug fixes

I discovered a semi-important tag processing bug last week--for some users, files that were imported before they added the PTR were not getting all the correct tag info and would not appear in some tag searches. I fixed the underlying file-tracking bug and have added a retroactive 'gap-filler' to this week's update. If you used the client for a long time but added/reset the PTR recently, you'll have more gaps so your update will take longer.

Another persistent issue has been the 'tiling artifacts' with my new tile-based image renderer, where at certain zooms an image would have one or more horizontal or vertical blurred/janked line of pixels. You'd notice it on cleaner vector images where a smooth curved black line would suddenly jag at one zoom. I had trouble reproducing this at first, but some users got it quite often, and with their help I was able to figure it out. I have rewritten the part of my renderer that was failing, and I think I now have the artifacts completely fixed for all 'normal' zooms. Let me know how you get on!

misc

Network job widgets have a little user-friendly update this week. I brushed up some of the status texts, made the text lay out better, and when the widget counts down waiting for something (like free bandwidth), if it doesn't get it because another downloader did, it says so before resetting the countdown clock.

A helpful user wrote a new darkmode style called OledBlack. Check it out under options->style!

I added OR predicates to Client API file search!

If you are an advanced user, you might like to read this idea I had for setting up multi-tag processing workflows: https://hydrusnetwork.github.io/hydrus/help/advanced_parents.html#parent_favourites I suddenly thought of it a couple weeks ago and it worked so well for me that I decided to write it up. It basically involves using local parent tags to group PTR tags into a pseudo-OR search that you can easily edit.
Open

R: 2 / I: 0 / P: 1

Version 452

https://youtube.com/watch?v=iePWOFOSl_U [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v452/Hydrus.Network.452.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v452/Hydrus.Network.452.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v452/Hydrus.Network.452.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v452/Hydrus.Network.452.-.Linux.-.Executable.tar.gz

I had an ok week. I ended up doing a lot of boring behind the scenes prep, but there is also some nice new quality of life. If you sync with the PTR, update will take a few minutes this week.

all misc this week

After a very long delay, I finally have shortcuts for seeking video back/forwards in the media viewer. New users will get ctrl+left/right to seek back 2.5 seconds or forwards 5 seconds. Existing users need to add their own--please check it under the 'media viewers - all' shortcut set. You can set whatever seek distance you like, and even set multiple with different distance jumps. Sorry for how long it took to get this in--I had to update my shortcut system first!

I did some behind the scenes tag work this week. The database can now handle larger 'tag as number' searches. Update will take a few minutes to load these bigger numbers into the fast search cache.

If you have a subscription that is not meant to completely sync (e.g. it pulls a sample from a gallery with unusual sort order, or you have the 'normal file limit' intentionally small), there is a new checkbox that suppresses the 'hey, there was a gap, click here to fill it in' popup windows.

full list

- misc:
- my 'simple' shortcut commands can now store additional variables. to start things off, I have finally added 'seek video' shortcut commands that have back/forwards and second+millisecond values. these should work on the native video viewer and mpv, audio and video. existing users will have to add their own (do it to the 'media viewers - all' set), but new users will get ctrl+left/right for 2.5s back and 5s forwards as the new defaults. let me know if you have any trouble!
- the maximum number tracked by 'tag as number' system predicate is expanded from -99999999->99999999 to -2^63 -> 2^63 - 1. tag caches will be regenerated on update to store these, it will take a few minutes. the input ui for the system predicate is temporarily limited to -/+2^31, but I'll expand it
- subscriptions now have a checkbox for 'do not worry about subscription gaps'. if you have a subscription that gets files randomly, or gets an intentionally small sample, this will disable the 'hey, there was a gap, click here to fill it in' popup messages
- you can now set negative values for the duplicate score weights in options->duplicates
- also added a weight for the 'nice ratio' duplicate comparison
Open

R: 3 / I: 0 / P: 1

Version 451

https://youtube.com/watch?v=Cv87NJ2xX0k [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v451/Hydrus.Network.451.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v451/Hydrus.Network.451.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v451/Hydrus.Network.451.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v451/Hydrus.Network.451.-.Linux.-.Executable.tar.gz

I had a great week cleaning code and fixing bugs. If you have a big database, it will take a minute to update this week.

all misc this week, most bug fixes

I fixed a critical bug in tag siblings. It was causing some tag siblings to be forgotten, particularly on local tag services, and particularly when a tag had a sibling removed and added (e.g. 'delete A->B, then add A->C') in the same transaction. I will keep working here and will trigger a new sibling reprocess in the future just as 450 did so we can fix more PTR issues.

The new content-based processing tracking had a couple more issues. Some Linux users, in particular, it just broke for, due to a SQLite version issue. I have fixed that, and I have fixed some issues it caused for IPFS. There are new unit tests to make sure this won't happen again.

I fixed an issue with sessions recently not saving thumbnail order correctly!

I fixed issues with some of the new Client API file search parameters not working right!

Big video files should import a bit faster, and will show more status updates as they do their work.

We have had more anti-virus problems in recent weeks. We'd hoped building on github would eliminate them, but it hasn't completely. A user helped me trace one issue to the Windows installer. We have 'corrected' it (believe it or not, it was removing the 'open help' checkbox on the final page of the install wizard, and yes the reasons for why this was causing problems are ridiculous), so with luck there will be fewer problems. Thank you for the reports, and let me know if you have more trouble!

full list

- stupid anti-virus thing:
- we have had several more anti-virus false positives just recently. we discovered that at least one testbed used by these companies was testing the 'open html help' checkbox in the installer, which then launched Edge on the testbed, and then launched the Windows Update process for Edge and Skype, which was somehow interacting with UAC and thus considered suspicious activity owned by the hydrus installer process, lmao. thereafter, it seems the installer exe's DNS requests were somehow being cross-connected with the client.exe scan as that was identified as connected with the installer. taking that checkbox out as a test produced a much cleaner scan. there is a limit to how much of this nonsense I will accomodate, but this week we are trying a release without that 'open help' link in the installer, let's see how it goes
- semi-related, I brushed up the install path message in the installer and clarified help->help will open the help in the first-start welcome popup message

R: 69 / I: 13 / P: 1

Bug Reports

Please feel free to submit any bug reports here.
Open

R: 3 / I: 0 / P: 1

Version 450

https://youtube.com/watch?v=fLLKwIfs1NM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v450/Hydrus.Network.450.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v450/Hydrus.Network.450.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v450/Hydrus.Network.450.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v450/Hydrus.Network.450.-.Linux.-.Executable.tar.gz

๐ŸŽ‰๐ŸŽ‰ MERRY 450! ๐ŸŽ‰๐ŸŽ‰

I had an ok week. Last week's experimental release went well, so I have polished that, and I otherwise caught up with a variety of small work.

If you sync with the PTR, update will take a couple of minutes.

all misc this week

So, the update storage change went ok! There were a couple of little sync bugs to clear up, but overall it works--tag repositories now track their processing progress by mappings, siblings and parents separately. You don't have to do anything, and this doesn't matter much for day to day work, but it allows for individualised reset and reprocessing. All 448 users will have their siblings and parents reset and reprocessed, which will take a couple of minutes to do on update, and about fifteen minutes on your next processing job to fill back in, and which should eliminate some bad siblings and parents due to years-old processing bugs that long term users have been dealing with (leaving only current bugs, which I am also working on). The reset will not delete any pending siblings or parents you have, so no worries if you have a bunch waiting to be uploaded.

Advanced Users: The PTR sibling and parent reset will however remove any siblings and parents you uploaded that were then denied by jannies (which your client would have added to itself anyway). Everyone is reset to a 'clean' sync with this change, so if you know you have a ton of surplus denied siblings you rely on, perhaps from years ago that we agreed I would deny on the PTR to help you hack in an overwrite in the old system, you might like to hold off updating and first figure out a PTR sibling/parent backup to a local service using tag migration.

I fixed some things with Mr Bones. His numbers are accurate to your 'my files' again, and he now talks about your total deleted files and also your earliest file import time. I divided the ugly growing stack of numbers into tabs, which I am sort of happy with, sort of not. People like to take screens of Mr Bones, but they have different preferences on what to show, so I may replace this with expand/collapse buttons or similar, so you can show everything if you want.

If you use the export files window to get a lot of files out, it now makes a progress popup. You can close the window while it is exporting and still see and cancel the job.

The Client API file search now supports file and tag domain selection (like the 'my files' and 'all known tags' buttons on a normal search page), and also file sort for searches. I know the Client API guys have been waiting on this, so with luck we should see some neat new search options in the Client API programs in the near future.

R: 189 / I: 12 / P: 2

Q&A Thread

Please feel free to ask questions about hydrus here.

As a reminder, the help and getting started guide is here:

https://hydrusnetwork.github.io/hydrus/help/index.html
Open

R: 6 / I: 0 / P: 2

Version 448

https://youtube.com/watch?v=VEwVAV3VPw4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v448/Hydrus.Network.448.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v448/Hydrus.Network.448.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v448/Hydrus.Network.448.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v448/Hydrus.Network.448.-.Linux.-.Executable.tar.gz

Hey, I am sorry, Endchan has been down or not allowed login/posting the past couple weeks when I needed to get release out, so there was a gap.

I had an ok couple of weeks. I was pretty ill in the middle, but I got some good work done overall. .wav files are now supported, PSD files get thumbnails, vacuum returns, and the Client API allows much cleverer search.

client api

I have added some features to the Client API. It was more complicated than I expected, so I couldn't get everything I wanted done, but I think this is a decent step forward.

First off, the main 'search for files' routine now supports many system predicates. This is thanks to a user who wrote a great system predicate text parser a long time ago. I regret I am only catching up with his work now, since it works great. I expect to roll it into normal autocomplete typing as well--letting you type 'system:width<500' and actually getting the full predicate object in the results list to select.

If you are working with the Client API, please check out the extended help here:

https://hydrusnetwork.github.io/hydrus/help/client_api.html#get_files_search_files

There's a giant list of the current supported inputs. You'll just be submitting system predicates as text, and it handles the rest. Please note that this is a complicated system, and while I have plenty of unit tests and so on, if you discover predicates that should parse but are giving errors or any other jank behaviour, please let me know!

Next step here is to add file sort and file/tag domain.

Next there's a routine that lets you add files to arbitrary pages, just like a thumbnail drag and drop:

https://hydrusnetwork.github.io/hydrus/help/client_api.html#manage_pages_add_files

This is limited to currently open pages for now, but I will add a command to create an empty file page so you can implement an external file importer page.

misc

.wav files are now supported! They should work fine in mpv as well.

Simple PSD files now get thumbnails! It turns out FFMPEG can figure this out as long as the PSD isn't too complicated, so I've done it like for .swf files--if it works, the PSD gets a nice thumbnail, and if it doesn't it gets the file default icon stretched to the correct ratio. When you update, all existing PSDs will be queued for a thumbnail regen, so they should sort themselves out in the background.

Thanks to profiles users sent in, I optimised some database code. Repository processing and large file deletes should be a little faster. I had a good look at some slow session save profiles--having hundreds of thousands of URLs in downloader pages currently eats a ton of CPU during session autosave--but the solution will require two rounds of significant work.

Database vacuum returns as a manual job. I disabled this a month or so ago as it was always a rude sledgehammer that never actually achieved all that much. Now there is some UI under database->db maintenance->review vacuum data that shows each database file separately with their current free space (i.e. what a vacuum will recover), whether it looks like you have enough space to vacuum, an estimate of vacuum time, and then the option to vacuum on a per file basis. If you recently deleted the PTR, please check it out, as you may be able to recover a whole ton of disk space.

I fixed Mr Bones! I knew I'd typo somewhere with the file service rewrite two weeks ago, and he got it. I hadn't realised how popular he was, so I've added him to my weekly test suite--it shouldn't happen again.
Open

R: 7 / I: 0 / P: 2

Version 446

https://youtube.com/watch?v=asojeparbK0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v446/Hydrus.Network.446.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v446/Hydrus.Network.446.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v446/Hydrus.Network.446.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v446/Hydrus.Network.446.-.Linux.-.Executable.tar.gz

I had an ok week. The client does not have a huge changelog this week, but the server has a neat privacy improvement.

The PTR is going to be doing a heap of maintenance this week. It will be 'busy' a lot, approximately one hour of busy and then three hours free. Please bear with it, and if you have a million mappings to upload, I recommend you just give it a break and come back later. I am not totally sure how long it will take. Best case it is a day, worst case it might take four or five.

null account

tl;dr: The hydrus server is now even more anon. You don't have to do anything.

As the PTR has moved to multiple accounts, we've had several good discussions about privacy. Separate accounts, despite being anon, could potentially leave a fingerprint of preferences on the server. If the server were to fall into bad hands many years from now, someone could mine those records--maybe mixed with one time you casually said 'yeah, I added that sibling'--and perhaps derive something from it.

There is no technical need to remember which account uploaded what long term, so now all hydrus servers completely anonymise their content after a certain duration, default 90 days. A new non-useable 'null' account takes possession of files, tag mappings, siblings, or parents after the delay, letting the original uploaders be forgotten. Janitors will still have time to work on recent account-based problems, but the historical record works just like the old shared public account: all merged together.

I have updated the privacy document in the help to talk a little about it. As long as you do not tag your own face in pictures or something, I think we are pretty great now, especially if you use a VPN.

https://hydrusnetwork.github.io/hydrus/help/privacy.html#account_history

It will take some time to anonymise the PTR or any other big server, as it has to go through its whole historical record to catch up. Please bear with it.
Open

R: 3 / I: 0 / P: 2

Version 445

https://youtube.com/watch?v=ulvvQHD4184 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v445/Hydrus.Network.445.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v445/Hydrus.Network.445.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v445/Hydrus.Network.445.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v445/Hydrus.Network.445.-.Linux.-.Executable.tar.gz

I had a great week mostly working on optimisations and cleanup. A big busy client running a lot of importers should be a little snappier today.

optimisations

Several users have had bad UI hangs recently, sometimes for several seconds. It is correlated with running many downloaders at once, so with their help I gathered some profiles of what was going on and trimmed and rearranged some of the ways downloaders and file imports work this week. There is now less stress on the database when lots of things are going on at once, and all the code here is a little more sensible for future improvements. I do not think I have fixed the hangs, but they may be less bad overall, or the hang may have been pushed to a specific trigger like file loads or similar.

So there is still more to do. The main problem I believe is that I designed the latest version of the downloader engine before we even had multiple downloaders per page. An assumed max of about twenty download queues is baked into the system, whereas many users may have a hundred or more sitting around, sometimes finished/paused, but in the current system each still taking up a little overhead CPU on certain update calls. A complete overhaul of this system is long overdue but will be a large job, so I'm going to focus on chipping away at the worst offenders in the meantime.

As a result, I have improved some of the profiling code. The 'callto' profile mode now records the UI-side of background jobs (when they publish their results, usually), and the 'review threads' debug dialog now shows detailed information on the newer job scheduler system, which I believe is being overwhelmed by micro downloader jobs in heavy clients. I hope these will help as I continue working with the users who have had trouble, so please let me know how you get on this week and we'll give it another round.

the rest

I fixed some crazy add/delete logic bugs in the filename tagging dialog and its 'tags just for selected files' list. Tag removes will stick better and work more precisely on the current selection.

If you upload tags to the PTR and notice some lag after it finishes, this should be fixed now. A safety routine that verifies everything is uploaded and counted correct was not working efficiently.

I fixed viewing extremely small images (like 1x1) in the media viewer. The new tiled renderer had a problem with zooms greater than 76800%, ha ha ha.

A bunch of sites with weird encodings (mostly old or japanese) should now work in the downloader system.

Added a link, https://github.com/GoAwayNow/Iwara-Hydrus, to Iwara-Hydrus, a userscript to simplify sending Iwara videos to Hydrus Network, to the Client API help.

If you are a Windows user, you should be able to run the client if it is installed on a network location again. This broke around v439, when we moved to the new github build. It was a build issue with some new libraries.
Open

R: 4 / I: 0 / P: 2

Version 444

https://youtube.com/watch?v=0OmOQs0ziSU [Embed]
This whole vid is great. Watching a ton of this stuff was a decent whack of my vacation.
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v444/Hydrus.Network.444.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v444/Hydrus.Network.444.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v444/Hydrus.Network.444.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v444/Hydrus.Network.444.-.Linux.-.Executable.tar.gz

I had an ok week getting back to speed. There's some new help regarding the new PTR accounts and I fixed some bugs.

accounts and help

If you use the PTR, you may have seen a note appear on the public account that it is now 'read only' and can now only download updates. If you would like to upload tags, siblings, or parents, please generate your own account either by clicking the popup button you get when trying to upload, or just hit up the PTR on services->manage services and click the easy button there. This pretty much completes the long project me and the guys running the PTR have been working on: moving all uploading users to their own still-anon-but-seperate accounts so the janitors can work through groups of petitions and undo future mistakes without it all being merged together in one account. I have given both of these pages a complete pass to represent the changes and updated screenshots etc... to be more modern:

https://hydrusnetwork.github.io/hydrus/help/access_keys.html
https://hydrusnetwork.github.io/hydrus/help/privacy.html

The privacy page now compiles the various conversations I have had with users over time and most recently about this account change, and my current understanding of exactly how private a hydrus repo is (really good), and what you can do to make it almost perfect, even against esoteric problems or an untrustworthy administrator (basically using a VPN is always a good idea when you are doing anything fun online, and don't tag your private photos on the PTR with your real name lol). I have tried to be comprehensive and even tried to imagine some advanced future problems, so it is a long read, but I'll try to keep updating all my thoughts to there so it can be a good reference point in future. If you are interested in this stuff, please read it and let me know what I have missed or if you have any other ideas!

Also, if you are a Linux user and get OOM (out of memory) crashes, please check the new thorough user-written guide linked in this new 'running' section here:

https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#running
https://hydrusnetwork.github.io/hydrus/help/Fixing_Hydrus_Random_Crashes_Under_Linux.md

all misc otherwise this week

I rewrote a heap of thread-UI object interactions that should improve stability on some Linux flavours and also get rid of the 'QtDeadWindow' errors some users have seen (usually in manage tags).

The new popup message button that fills in subscription gaps would sometimes not assign the correct downloader (usually when the button was opening a new page). This should be fixed!

A variety of error reporting is more reliable and presents better text.

An odd error when loading the downloader .png files is fixed, so if you were unable to import new downloaders in the past few weeks, please try again and let me know how you get on!
Open

R: 3 / I: 0 / P: 2

Version 443

https://youtube.com/watch?v=NgYIIPszZjA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v443/Hydrus.Network.443.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v443/Hydrus.Network.443.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v443-macos/Hydrus.Network.443-macos.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v443/Hydrus.Network.443.-.Linux.-.Executable.tar.gz

I had a great week doing nice cleanup and quality of life work.

Hey, we had a problem getting the macOS release to build this week. The macOS link above goes to a build using a simpler and faster method. It should work fine, but please let me know if you have any trouble. As always, back up before you update!

highlights

Popup messages can now launch complex jobs from a button. The first I've added is when a subscription hits its 'periodic' file limit. The situation itself is now better explained, and a button on the popup will create a new downloader page with the specific query set up with an appropriate file limit to fill in the gap. The second is if you try to upload some content to a repository that your account does not have permission for (this is affecting sibling- and parent-uploading PTR users as the shared public account is changing), the popup message that talks about the issue now has a button that takes you straight to the manage services panel for the service and starts automatic account creation.

Subs should now be more careful about determining when they have 'caught up' to a previous sync. Small initial file limits are respected more, and the 'caught up' check is now more precise with sites that can give multiple files per URL or very large gallery pages.

I gave options->speed and memory a full pass. The layout is less crushed and has more explanation, the options all apply without needing a client restart, and the new, previously hardcoded cache/prefetch thresholds are now exposed and explained. There's a neat thing that gives an example resolution of what will be cached or prefetched, like 'about a 7,245x4,075 image', that changes as you fiddle with the controls.

The client has recently had worse UI lag. After working with some users, the biggest problems seemed to come in a session with lots of downloaders. I traced the cause of the lag and believe I have eliminated it. If you have had lag recently, a second or two every now and then, please let me know how things are now.

If you use the Client API a lot while the client is minimised, you can now have it explicitly prohibit 'idle mode' while it is working under options->maintenance and processing.
Open

R: 5 / I: 0 / P: 2

Version 442

https://youtube.com/watch?v=bpEFn3MFyfA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v442/Hydrus.Network.442.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v442/Hydrus.Network.442.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v442/Hydrus.Network.442.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v442/Hydrus.Network.442.-.Linux.-.Executable.tar.gz

I had a great week. An important part of GUI Sessions is overhauled, which should save a lot of hard drive time for larger clients.

gui sessions

I always encourage a backup before you update, but this week it matters more than normal. If you have a client with large sessions with many important things set up, make sure you have a backup done before you update! I feel good about the code, and I try to save data on various failures, but if your situation gives errors for an unforeseen reason, having the backup ready reduces headaches all around!

Like the subscriptions and network objects breakups I've done in the past year, I 'broke up' the monolithic GUI Session object this week. Now, when your session has changes, only those pages that have changed will be saved, saving a ton of CPU and HDD write I/O. Furthermore, sessions that share duplicate pages (this happens all the time with session backups), can now share that stored page, saving a bunch of hard drive space too. Like with subscriptions, some users are pushing multiple gigabytes of session storage total, so there is a good amount of work to save here.

You don't have to do anything here. Everything works the same on the front end, and all your existing sessions will be converted on update. Your client should be a little less laggy at times, and client shutdown should be a bit faster.

If any of your old sessions fail to load or convert, a backup will be made so we can check it out later. Let me know if you have any trouble!

Advanced stuff:

Another benefit is the old limit of 'sessions fail to save at about 500k session weight' now applies to pages individually. Please don't immediately try to nuke your sessions with five million new things, but if you do end up with a big session, let me know how other performance works out for you. Now this bottleneck is gone, we'll start hitting new ones. I believe the next biggest vulnerability is thread starvation with many simultaneous downloaders, so again please don't paste-spam a hundred now queries (for now).

If you have been tracking session weight (under the pages menu), I am rebalancing the weights. Before, the weight was file = 1, URL = 1, but after all our research into this, I am setting it to file = 1, URL = 20. In general, I think a page will fail to save at the new weight of about 10 million. If you are in advanced mode, you can now see each page's weight on page tab right-clicks. Let's get a new feeling for IRL distribution here, and we can aim for the next optimisation (I suspect it'll eventually be a downloader-page breakup, storing every query or watcher as a separate object). Since URLs seem to be the real killer, too, see if you can spread bigger downloads across multiple download pages and try to clear out larger completed queries when you can.
Open

R: 2 / I: 0 / P: 2

Version 441

https://youtube.com/watch?v=EJLNLWv-nmM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v441/Hydrus.Network.441.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v441/Hydrus.Network.441.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v441/Hydrus.Network.441.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v441/Hydrus.Network.441.-.Linux.-.Executable.tar.gz

I had an ok week. Not as much as I wanted, but there are some nice Client API improvements.

all misc this week

The test builds from last week seem to work ok, so they are now master. The built clients now use Python 3.8, and the security libraries (like OpenSSL) are all much newer--and will reliably stay up to date in future--so a whole bunch of things across the client should have slightly better performance. There are no special install instructions, they seem to work on an existing install just as normal. Let me know if you do run into any problems!

I fixed some more bad tiles calculations for the new tiled image renderer. Some files that seem to have little black lines on an edge at some zooms, or previews that just turn up black, should be fixed! Error reporting is also nicer.

The Client API can now do a couple more things. Particularly, it can now set your client's global User-Agent, which should help fix some difficult CDN and login problems in future. Please watch this space.

For advanced users, if you have help->advanced mode on, then setting a namespace file sort now allows you to choose which 'tag context' the sort works on. If you hide certain tags in single or multiple media view (as set in tags->manage tag display and search), then those hidden tags will not count for the sort. This is obviously advanced, so if you hadn't thought of it, you can just set 'display tags' to keep 'normal' behaviour.

full list

- misc:
- after successful testing, all the master builds are now made on github rather than my home dev situation. the clients now work off python 3.8, and several security libraries (e.g. OpenSSL) are now always going to be latest, so there should be several quiet performance and reliability improvements across the program. there are no special install instructions--normal update seems to go fine--but let me know if you do have any trouble. big thanks to the user who did the leg work on developing the workflow build scripts here
- if you are in advanced mode, namespace file sorting now allows you to set the 'tag display context' on which it will sort. this appears as a new menu button or a button list selection dialog wherever you edit namespace file sorts. if you are not in advanced mode, the default is the 'display tags' I switched to last week (i.e. before any tags are hidden by your tag display options)
- namespace sort has some related code cleanup. the 'defaults' object is updated and moved to the newer options object
- the new tiled renderer now checks for rounding errors in zoom calc, which in some cases was giving a single extra (non-existing) native pixel row or column on rightmost or bottommost tile samples
- the new tiled renderer now double-checks clip regions for validity before attempting to crop
- improved the reported error information when a tile fails to render
- when pasting an uneven number of tags into manage siblings/parents, the error is now a nicer popup dialog. I'm pursuing a related error here--if you get this a bunch, please let me know what more info you discover
- when repositories fail to fetch the update hashes to process, they now force a metadata resync. any processing error should force a metadata resync now
- added a default url class for the new pixiv _artist_ page format
- fixed a recent typo bug with ipfs pinning

R: 1 / I: 0 / P: 2

unable to keep my tags visible and overall finicky

whattup,

I've been fiddling with hydrus for a bit but cannot figure out how to keep my tags visible and usable at all times. I have two tags so far, but expect to use dozens, if not hundreds in total in the future.

The whole thing isn't easy for me introductory tutorial that I've gone over so far has basically been "select image, press f3, do ur tag stuff".

Can I see all my own created tags without selecting an image and pressing f3 and be able to edit/delete them? Also, how do I keep tags in the "selection tags" section at all times?

Thanks in advance.
Open

R: 3 / I: 0 / P: 2

Version 440

https://youtube.com/watch?v=mO1j6xx2HhQ [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v440/Hydrus.Network.440.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v440/Hydrus.Network.440.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v440/Hydrus.Network.440.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v440/Hydrus.Network.440.-.Linux.-.Executable.tar.gz

I had an unfortunately short week, but I did some good work. The tiled renderer has nice fixes.

tiled renderer

I regret the tiled renderer, while good most of the time, had crashes when it did go wrong. To stop this with any new errors that might pop up, the whole block now has an additional layer of error catching around it. If a tile fails to render for any reason, you now get a black square, and if some coordinate space cannot be calculated or a portion of the image is truncated, the system will now print some errors but otherwise ignore it.

A particular problem several users encountered was legacy images that have EXIF rotation metadata but were imported years ago when the client did not understand this. Therefore, hydrus thought some old image was (600x900) when it then loaded (900x600). In the old system, you would have had a weird stretch, maybe a borked rotation, but in my new tiled system it would try to draw tiles that didn't exist, causing our errors-then-crashes. The client now recognises this situation, gives you a popup, and automatically schedules metadata regeneration maintenance for the file.

some misc

You can now set a custom 'namespace' file sort (the 'series-creator-volume-chapter-page' stuff) right on a page. Just click the new 'custom' menu entry and you can type whatever you like. It should save through your session and be easy to edit again. This is prep for some better edit UI here and increased sort/collect control, so if you do a lot of namespace sorting, let me know how you get on!

I prototyped a new 'delete lock' mode, which prohibits deletion of files if they match a criteria. I am starting with if a file is archived. You can turn this mode on under options->files and trash. It mostly just ignores deletes at the moment, but in future I will improve feedback, and maybe have a padlock icon or something, and eventually attach my planned 'metadata conditional' object to it so you'll be able to delete-lock all pngs, or all files with more than four tags, or whatever you can think of.
Open

R: 3 / I: 0 / P: 3

Version 439

https://youtube.com/watch?v=CyPKxkH3vB8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v439/Hydrus.Network.439.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v439/Hydrus.Network.439.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v439/Hydrus.Network.439.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v439/Hydrus.Network.439.-.Linux.-.Executable.tar.gz

I had an ok week. The new tiled renderer is improved.

tiled renderer

The new image drawing system generally worked well! There were a couple of bugs, and it still has some limitations, but in general it really improved zoom and precache performance.

For the bugs, first of all, there was a rare crash, I think triggered by loading a very unlucky coincidence of tile and image size. Then clipboard bitmap copy threw an error, tiny images could not deal with extremely small zoom, and clients under heavy load could sometimes have trouble initialising the viewer. I have fixed them all, but let me know if you have any more trouble!

There was also a problem with PyQt5, an alternative version of the Qt UI library that some 'running from source' users use. It was an object handling difference between PyQt5 and PySide2 that broke the tile caching system. I think I have fixed it, so if you are running PyQt5, please give this version a go.

Beyond bugs, there were tiling artifacts visible at higher zooms. Essentially, where the tiles lined up, there were small disagreements in resize math, resulting in little lines of mismatching colour gradients along tile borders. I worked on the tiling algorithm and have significantly mitigated the problem--I mostly only see artifacts at extreme zooms now, about 2000%.

Since people are suddenly zooming more, users who have mouse-centered zooming were having more images accidentally flying off screen too. I've hacked in off-screen rescue after a zoom, sliding it back to the nearest border, so the image should always stay in view. If people like it, I may patch this in for all media for dragging events too. There's not much need for non-visible media, and when it does happen it can sometimes be a pain dragging around to find where it went.

I hope this basically makes the tile render a complete '1.0' now. In the future, I would like to rejigger some of the virtual geometry, since at the moment a limit in Qt means I cannot zoom higher than a 'virtual' 32,768x32,768 canvas (e.g. 4k at about 800% zoom). I'll also replicate the tiling for my native Animation widget, which displays gifs and video when mpv is not available.
Open

R: 4 / I: 0 / P: 3

Version 438

https://youtube.com/watch?v=LE8QKcriHH4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v438/Hydrus.Network.438.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v438/Hydrus.Network.438.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v438/Hydrus.Network.438.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v438/Hydrus.Network.438.-.Linux.-.Executable.tar.gz

Hey, this causes errors if you are running from source and using PyQt5 (PySide2 is fine)! All the releases above are PySide2, so they are ok! I will fix this for next week, so if you are source+QtPy5, please hold off for now.

I had a great week overhauling the media viewer's image rendering. Zooming and navigation should be a lot smoother now!

image tiles

tl;dr: the media viewer now zooms and navigates with less lag and flicker

Zooming in a long way, particularly for large images, has been pretty hellish for as long as the program has existed. Historically, the client drew the whole image in memory at the zoom you desired so it could quickly show you the bit it needed on screen. Of course this meant zooming in to 400% on anything above 4k was suddenly taking a very long time to set up and eating a lot of memory to do it. As images have naturally grown over time, the problem has occurred more often and was starting to affect other systems.

My plan to fix this has been to break the image into tiles that then render on demand. The parts of the image off-screen are never drawn, saving CPU and memory and allowing arbitrary zoom. This is a significantly more complicated idea, and rewriting the whole rendering pipeline was always expected to be a multi-week 'big job'. I originally planned to just optimise and tweak the secondary systems and add in some sanity brakes this week, but I ran a couple of small tiling tests and realised if I went bonkers it would be possible to hack in a prototype. So I did!

In the media viewer, images now draw in tiles. It works a little like a browseable satellite map, where when you zoom in and pan about you see squares of data fading in (except in hydrus they appear instantly). You should now be able to zoom in as far as you like on an image pretty quick and you won't have any sudden memory needs.

Furthermore, I have written a cache for these image tiles. This saves CPU when revisiting different images or zooms, so when you flick back and forth between two normal things, it should now be instant! It still takes 20-200ms to view or zoom most images the first time, but going back to that view or zoom within a minute or so should be really smooth. The cache starts at a healthy 256MB this week. I think that will cover most users very well (in screen real estate, it works out to about 35 x 1080p worth of tiles), but you can alter it under the settings at options->speed and memory.

And I did some misc work improving the rendering pre-fetch logic when you browse in the media viewer. Huge files won't stomp all over the image renderer cache any more, which should make browsing through a series of giant images far less jank. If you are feeling advanced, you can now edit the prefetch timing and distance settings too, also under options->speed and memory.
Open

R: 3 / I: 0 / P: 3

Version 437

https://youtube.com/watch?v=FfBdUvVQpNQ [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v437/Hydrus.Network.437.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v437/Hydrus.Network.437.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v437/Hydrus.Network.437.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v437/Hydrus.Network.437.-.Linux.-.Executable.tar.gz

I had a good week mostly fixing bugs and optimising. It will take a couple of seconds to update this week.

all misc this week

I reduced a heap of UI lag on clients that have pages with a lot of collected (like 'collect by creator') media. If you often have five or ten thousand files collected and you noticed your client was getting choppy just when running some downloaders, I hope this improves things. Let me know how you get on!

I started real work on multiple local file services. Most of it is boring behind the scenes stuff, but as part of it, I overhauled the trash system this week. A heap of logic is improved, it is ready for more than one 'my files' service, and now hydrus remembers when files are deleted. Delete timestamps have never been recorded clientside, and unfortunately we cannot recover old information retroactively, but it is stored for all deletes from now on. Whenever the client wants to say 'this file was deleted', it should now have 'at an unknown time' or a nicer '3 days ago' suffix.

A second neat thing with the improved deleted files storage is I hope in the nearish future to let you search deleted files. This is a rare, clever query, like 'all known files', but there will be some kind of button you can press to flip your 'my files' or 'all local files' search to go through what has ever been removed from them. 'system:file service' will get similar improvements.

To reduce confusion, I renamed some hydrus network concepts across the program. The 'access key' (secret password for account), 'account key' (account identifier), and 'registration key' (one time token to create an account) are now known as 'access key' (i.e. no change), 'account id', and 'registration token'. There is more work to do here, particularly improving server setup and account management workflows to suit the user (rather than my technical ease), so I will keep at it.

In a related topic, the PTR is updating its accounts. The public account is moving more towards a 'read-only' account, and accounts that can upload siblings and parents (and perhaps tag petitions, eventually) will be individual to you and freely auto-creatable in manage services. This is mostly an attempt to make janitorial decisions easier and more accurate, since at the moment everything on that side is merged due to the shared account. Permissions have not been used much in hydrus network yet, and the workflows and user notifications here are bad and buggy. Please bear with me as I iron out the problems and make it all nicer to use.
Open

R: 3 / I: 0 / P: 3

Version 436

https://youtube.com/watch?v=i_u3hpYMySk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v436/Hydrus.Network.436.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v436/Hydrus.Network.436.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v436/Hydrus.Network.436.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v436/Hydrus.Network.436.-.Linux.-.Executable.tar.gz

I had a great few days mostly cleaning and fixing things. If you sync with the PTR, update will take a minute this week.

macos release polish

I cleaned up the new macOS release. It seems to have launched and otherwise generally worked last week, but there was a bug in finding the specific database location macOS users are used to. Without the '--d' launch parameter, it was creating an empty new db inside the app, in the 'db' dir hydrus would normally use (and the really old App used to use, if you remember that), and hence would say 'hey, this looks like the first time you are running the program...' on boot. I have fixed the 'I am running in an app' detection and the ~/Library/Hydrus database path calculation routine, so everything should be back to normal.

It also has the old readme and Applications shortcut in the dmg, and the filename should be fixed too. I expect this to be the only macOS release I put out from now on. Let me know if you have any more trouble!

miscount fix

Last week, I made the number in the 'pending (1,234)' menu title add up in a more efficient way. Rather than counting raw mapping rows every time, it uses a table of pre-computed numbers, the same used for autocomplete results. It turns out there were some legacy (from a long time ago) miscount bugs in there for some users. This resulted in a 'sticky' number that would not go away even after committing. A maintenance routine exists to fix this, but it is a sledgehammer when we need a scalpel.

So, I have written a maintenance routine to regen this pending data efficiently and correct these old bugs. It is basically the same as I did a few months ago with the 'display' caches during the siblings and parents work, but for a deeper level of tags. It will be run on update, along with a new thing that forces the menu's count to regen, both of which can now be accessed from database->regenerate menu in case we need them again in future. If you sync with the PTR, it may take a minute or so to finish.

I hope this will fix the issue completely, but if you still have a bad count, or if your count drifts off zero again over time, please let me know!

underscores

After discussion with some users, I have added an experimental setting to options->tag presentation that replaces all underscore characters in tags with space characters, as long as you are in 'front-facing' UI like regular search pages or the media viewer. It works on the same system as the 'hide namespace' option--and siblings--in that you still see the raw truth in manage tags and other edit locations.

This setting is experimental since it will add a bit of CPU lag to tag presentation and may result in some seemingly duplicate rows. I have long planned to fix the underscore issue with a really nice system, but I was convinced that adding a hacky system in the meantime would be a good thing to play with. If you care about this issue, give it a go and let me know if you run into any problems.
Open

R: 3 / I: 0 / P: 3

Version 435

https://youtube.com/watch?v=-ZdLbiF9gCo [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v435/Hydrus.Network.435.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v435/Hydrus.Network.435.-.Windows.-.Installer.exe
macOS
NEW app: https://github.com/hydrusnetwork/hydrus/releases/download/v435/HydrusNetwork-v435.dmg
OLD app: https://github.com/hydrusnetwork/hydrus/releases/download/v435/Hydrus.Network.435.-.macOS.old.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v435/Hydrus.Network.435.-.Linux.-.Executable.tar.gz

I had a good week. I continued last week's server work and have a macOS Big Sur build available for testing.

new macOS build

This was delayed far longer than I wanted, but I think I have it figured out now. Thanks to work from Suika and ReAnzu on github figuring out the script, the macOS build is now put together on github itself, on Catalina. This new build thus works on Big Sur, and should generally have less compatibility jank on any newer macOS machine. I have included my old 10.12-built release as well today, just in case I messed something up, but if you are a macOS person, please give the new one a try and let me know how you get on.

Assuming this works out ok for the majority of people, I'll be moving to it as the official macOS build. Anyone who is still on an old macOS will have to run from source to stay updated with hydrus. The new release is just out of testing, so I'll clean up the filename and add the readme.rtf and so on so it looks prettier. I will also explore reducing the size (300MB vs 137MB wew), but I am told this may be unavoidable due to some new macOS thing that bundles both old and new versions of libraries in the App.

editable repository update periods

Just in case, it is probably a good idea to back up your server before you try this!

Hydrus repositories now have an editable update period! This was an oft-requested feature for users running a server on a LAN or between just a few friends, so if this is you, please give it a go. There is a new 'edit options' permission for accounts, which any account with 'manage account types' (admins basically) will inherit on update. If you have this, the 'admin services' menu will have a new 'edit update period' entry, which launches a simple time delta widget where you can change it.

Changes occur immediately. The server will create any updates that are now due, and your client will do a resync to keep up to date with the new numbers, so if you have 'review services' open when you make the change, you should see everything update in realtime. Other clients that sync with the server will catch up with the new check times when they next do their normal sync, unless they too force a resync with the new 'review services' button I added last week.

You can change the update several times, up or down, and it shouldn't be a big deal--the client no longer cares if one set of updates were short or long--so please feel free to play around with this a bit, just don't go crazy with super short or long times. The minimum is 10 minutes, but I recommend you only go down to about 60 minutes if you really want to be that fast, at least as we experiment with this. The system was originally built with 100,000 seconds in mind, so I think super short periods will add overhead bloat over time (not to mention, you only see content after processing, which atm only happens in idle time anyway).
Open

R: 2 / I: 0 / P: 3

Version 434

https://youtube.com/watch?v=f1nvZ74OY_I [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v434/Hydrus.Network.434.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v434/Hydrus.Network.434.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v434/Hydrus.Network.434.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v434/Hydrus.Network.434.-.Linux.-.Executable.tar.gz

I had a great three weeks working on long overdue network updates. Most of the changes this week restore old account management UI for server admins, but there are a couple of fixes and improvements for regular users as well.

The network version goes up to 20 today. Clients and servers can only talk to each other if they are on the same version, so if you want to talk to the PTR, you will want to update at some point. No rush, but if you try to talk to the server on an older client, you will get a polite error message.

regular user stuff

On review services, you can now pause repository account sync, update downloading, and update processing separately. If you don't want to process right now but still want the client to be grabbing update files for later, or you want to pause all service network traffice for a while but still process your backlog, it should now be easy to set up.

I fixed an annoying bug where going from 'random' file sort to one with asc/desc sort order by loading a favourite search would always default the asc/desc to the top value. Guess what hydev's default sort is.

Server administrators can now set messages to accounts. If you get a message, your client should get the update within a day or two, at which point it will appear in a popup box and be viewable on review services.

Server administrators can now set some account types to be user-creatable--no need to mess around with registration keys or access keys yourself, the client can fetch them for you from the server. A new button on the manage services edit panel checks this and can do it for you. I understand the PTR is likely to move to this in future for accounts with sibling and parent petition permissions, since grouping these by account will be useful for janny petition workflow. Having separate accounts for these complicated uploads will also allow better account management (including the above rudimentary messaging) for users who make common errors with sibling or parent submissions.

I believe I fixed a bug in repository processing that caused some newer users to get stuck with siblings and other data that was later deleted. This should not happen again, and I am now planning ways to efficiently fix the problem retroactively. (If you were hit by this and want the non-efficient fix, hit 'reset processing->reprocess content' for the repo under review services).

Some users recently got some scary errors about an invalid repository update file. This was the result of the client becoming confused about which update files it should have for a repository. I am still investigating why this happened, but to fix the immediate issue I have added a maintenance routine (under the new review services->reset downloading) that triggers a full resync of update metadata and delete surplus records. This function will fire automatically in future if the service encounters borked update files of any kind. If you got these errors, please try firing it and let me know how you get on.
Open

R: 5 / I: 0 / P: 3

Version 433

https://youtube.com/watch?v=twOFs6zJq8Q [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v433/Hydrus.Network.433.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v433/Hydrus.Network.433.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v433/Hydrus.Network.433.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v433/Hydrus.Network.433.-.Linux.-.Executable.tar.gz

I had a good week doing simple cleanup and fixes. This is intended to be a 'clean' release. There will be no release for about three weeks.

all misc this week

Thanks to the work of a user (who was also very patient as I slowly added the features he needed), we now have a new native twitter downloader! It gets video now! (Including gif, which on twitter is just a small mp4!) It is called 'twitter syndication' and uses a new access method. Its main limitation seems to be that its search can only reach about 500 tweets into the past. Please check it out, and if it works well for you, please move your subscriptions over from nitter to save them some bandwidth. This is the first version, so it may well have future updates.

I have written a new 'scrollable' menu button, which works like the file sort selector--instead of a dropdown, you get a menu, and you can scroll through it in either direction with your mouse. I have converted several old small choice dropdowns to this new button: file sort ascending/descending, collect-by 'collect/leave unmatched', and all the new tag sort dropdowns I added last week. Try putting your mouse over and scrolling them! This is a little thing, but I hope it smooths out the workflow here. I'll likely use it on more things.

Some more mouse changes: Middle-clicking on the collect-by dropdown now clears it. Middle-clicking or right-clicking the 'page tab bar' to open the 'new page' chooser or the page right-click menu should now only work on the tab bar's empty greyspace (previously, this was confusingly happening on a click on greyspace in any child panel, like some random downloader page panel).

Namespace file sort (the 'series-creator-title...' stuff) now support a-z/z-a ordering. Files that have no match are still counted as 'less than a' in a-z terms, but I can change this stuff easier these days. I don't use this myself much, so if you do, let me know what you'd like in future.

Network job controls (the little panel that shows your downloads) now let you edit their active job's bandwidth rules from the cog menu! It will show all the network contexts and which are currently waiting on bandwidth. If the job is using the 'default' rules, it will say so and allow you to set specific ones just for that domain.
Open

R: 3 / I: 0 / P: 3

Version 432

https://youtube.com/watch?v=_ZDUHbIVaz8 [Embed]
^ After years of seeing all sorts of smaller edits using this footage, I finally discovered this source vid this week. If you like fine art, please watch it in full, it is eight minutes of unbroken kino. It gets a bit heavy at the end, not appropriate for kids. If you don't like old art, no worries, I was just chuffed this week to have finally found this thing.

windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v432/Hydrus.Network.432.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v432/Hydrus.Network.432.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v432/Hydrus.Network.432.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v432/Hydrus.Network.432.-.Linux.-.Executable.tar.gz

I had a great week. There are a bunch of different little fixes and improvements this week, and the tag sort dropdown is now easier to use.

tag sort

Rather than a big list, the tag sort dropdown is now a dynamic control that splits the different variables (sort type, asc/desc, group by) into different mini-dropdowns that show and hide when available. Confusing labels like 'lexicographic' and 'ignore namespace' are replaced with clean 'sort by tag/subtag/count'. I hope it is a bit easier to work with now. Give it a go and let me know what you think.

If you don't like your default tag sort, you can set it under options->tags. Your existing default should be converted to the new format on update.

I also added the option to sort by sibling text or the actual tag text when you have the manage tags dialog open.

There is more work to do. One thing I don't like is if, say, on the asc/desc dropdown, you have 'a-z' selected, you can hover your mouse over and scroll down to get 'z-a', but you can't scroll up to get it. I prefer to have scroll work like 'flip' when there are only two values, where either scroll direction will make a change, so I'll figure that out, and in the file sort/collect controls, so there is even less to think about when you edit. I'll also see about saving the tag sort in the session.

I also updated the back-end of tag sorting here, so it'll be easier to update in future. Some users have asked for custom namespace sorts (e.g. creator, character, series, then the rest (a-z), then unnamespaced), so I'd like to start thinking about that properly.

misc

As some users have not realised they were getting a big session that was lagging them, the client now makes a once-per-boot gentle warning popup if your session exceeds 500k weight. If you cannot conveniently reduce your session size, you can turn this warning off under options->gui pages.

Subscriptions now work more efficiently with a very small 'first run' file limit and a larger 'normal' file limit. Previously, the second run was often charging right through the handful of files caught on first sync and getting more than it should, but now it should better recognise it has 'caught up' and stop at the right position.
Open

R: 3 / I: 0 / P: 3

Version 431

https://youtube.com/watch?v=CU5NKzKiAGU [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v431/Hydrus.Network.431.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v431/Hydrus.Network.431.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v431/Hydrus.Network.431.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v431/Hydrus.Network.431.-.Linux.-.Executable.tar.gz

I had a challenging week, but I got some decent fixes in.

all misc this week

Pretty much everything was little fixes this week. Nothing hugely significant, but check the changelog if you have been waiting on something.

The e621 parser now pulls 'lore' tags, which seems to be some new 'canonical' tag. I have put them in 'lore' namespace, so you can exclude them easily if you want. One user suggested they are useful for certain blacklists.

For advanced users, I added a 'sorter' to string processing. It has a neat thing where you can sort by a regex'd substring of the main string. I was happy about this, but then I realised this morning that it is only half of the job I wanted to do. I will add a String Slicer next week so you can just sample the first item or the last three items or whatever you like, once you have your sort figured out.

full list

- misc:
- when parents are hidden in the edit/write taglists (e.g. in manage tags), there is now a '(n parents)' suffix
- thread watchers that are DEAD or 404 but still have files downloading now report 'working' status until that is done
- search terms with ';' like 'steins;gate' should now work in downloaders. sorry for the trouble!
- fixed an issue where un-ideal tags were sometimes becoming non-searchable when they were entirely replaced in manage tags with their 'ideal' siblings (i.e. their autocomplete count went to 0). this was due to overzealous deletion in the new tag definitions cache not filtering out sibling/parent chain members. a small routine will run on update to resynchronise affected tags
- fixed an issue when loading up files in the main 'import files' dialog where a critical error (as opposed to a nice 'couldn't figure it out, sorry') in mime detection would cause the whole job to hang
- that main 'import files' dialog now counts 'missing' files separately in the error count
- fixed tags not updating on the filename tagging dialog when double-clicking to remove from the simple taglists
- fixed the sort on the manage tags dialog's suggestion taglists--they now preserve their original sort, rather than alphabetising once sibling/parent data is populated

R: 0 / I: 0 / P: 3

e621

tried to login after changing password and I get this message about remember cookie missing in post login step
it says in logged in when i look at the manage logins
Open

R: 3 / I: 0 / P: 4

Version 430

https://youtube.com/watch?v=Giy5iQ9gGvc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v430/Hydrus.Network.430.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v430/Hydrus.Network.430.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v430/Hydrus.Network.430.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v430/Hydrus.Network.430.-.Linux.-.Executable.tar.gz

I had an ok week. I mostly fixed and improved things. If you sync with the PTR, it will take a minute or two to update.

all misc this week

When I did the autocomplete lookup speedup recently, we unfortunately lost the ability to match unusual characters with short tag inputs. Entering 'a' would not return a match of '/a/', and in fact there was no way to look that tag up without some slow cleverness like wildcards. I have fixed this this week. If you sync with the PTR, it will take a minute or two to update while a small new cache is constructed.

You can now hide/show the 'hanging' parents in taglists (in manage tags dialogs etc...) from the list right-click menu. You can also set in options->tags whether they should start shown/hidden for normal taglists and autocomplete dropdown results.

Taglists that show '(displays as xxx)' sibling info now sort by that sibling for lexicographic purposes. I had thought to make this optional, but when I tried it out, it felt great to me, so I left it as default. Let me know what you think.

I fixed an issue that meant some texts across the program were staying blank for a few seconds when they should have been updating. This mostly affected the 'highlighted downloader' panel in gallery download pages. This was personally driving me nuts IRL.

I fixed the tumblr downloader. Sorry for the problems here, I misunderstood initial reports. This was due to a logical change in hydrus that just happened to hit tumblr's URL format. If you have tumblr subscriptions, please go into them and see if any recent URLs have the status 'Found 0 new URLs.'. Right click them and say to 'try again' and they should fix up.
Open

R: 6 / I: 1 / P: 4

Version 429

https://youtube.com/watch?v=ezC4ybLJsPM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v429/Hydrus.Network.429.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v429/Hydrus.Network.429.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v429/Hydrus.Network.429.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v429/Hydrus.Network.429.-.Linux.-.Executable.tar.gz

I had a good simple week just working on smaller jobs and cleanup. If you have a large client, it may take a minute for you to update this week.

all misc this week

The new taglists seem to have gone generally well. There was one display/update error that hit some users, which I have fixed, along with some little menu logic cleanup. I will work on more display and sort options for them in future.

Since I added 'pending' status for thread watchers last week, you may have seen some apparently stuck on that status. This was a harmless legacy issue, just old check jobs that were never closed out correctly, which I have fixed. I also cut out the 'checking in' prefix from the waiting-to-check status text, let's see if that fits better.

Dialogs with multi-column lists should be less of a pain this week. I think I fixed the issue where the last column could sometimes shrink to a few pixels on the next dialog launch, and I think I also fixed the issue where columns would sometimes get a few pixels thinner every launch. Just set them once more to the size you want, and they should stick now!

'.clip' files, those for Clip Studio Paint, are now recognised and importable. Too complicated for me to support thumbs, but they have their own icon. I cleaned up the behaviour of the increasingly tall filetype selection widget, so I am happy to start adding simple new filetypes for a bit. If you would like a new media filetype added to hydrus, please send an example of that file to me with a bit of info about it. Nothing too obscure, please, and I can't promise I can add everything quick, but many are actually fairly simple. I'd like to see about .sai and .wav, and maybe start on an xml scanner so I can at recognise things like .svg.

I took a bit of time to check out animated webp this week. Unfortunately, gathering duration and frame info about webps seems to be much more difficult than actually making webps, so I am afraid we will have to wait for an update to FFMPEG or one of the other libraries we rely on to get animated webps going.

A new cache speeds up a bunch of file database lookups this week. If you have a large client, it may take a minute to update as this is populated.

R: 25 / I: 3 / P: 4

Suggestion Threads

Drop your suggestions here
http://archive.fo/heSey
Open

R: 11 / I: 0 / P: 4

Version 428

https://youtube.com/watch?v=RNPnBVHSeZc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v428/Hydrus.Network.428.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v428/Hydrus.Network.428.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v428/Hydrus.Network.428.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v428/Hydrus.Network.428.-.Linux.-.Executable.tar.gz

I had a good couple weeks working on the taglist code and some other jobs.

If you are on Windows and use the 'extract' release, you may want to do a 'clean' install this week. Extra notes below.

taglists

So, I took some time to make taglists work a lot cleaner behind the scenes and support more types of data. A heap of code is cleaner, and various small logical problems related to menus are fixed. The tag right-click menu is also more compact, quicker to see and do what you want.

The main benefits though are in the manage tags dialog. Now, the '(will display as xxx)' sibling suffix colours in the correct namespace for the sibling, and parents 'hang' underneath all tags in all the lists. It is now much easier to see why a parent or sibling is appearing for a file.

This is a first attempt. I really like how these basically work, but it can get a bit busy with many tags. With the cleaner code, it will be much easier to expand in future. I expect to add 'expand/collapse parents' settings and more sorts, and maybe shade parents a bit transparent, in the coming weeks. Please let me know how it works for you IRL and I'll keep working.

the rest

The main nitter site seems to be overloaded. They have a bunch of mirrors listed here: https://github.com/zedeus/nitter/wiki/Instances

I picked two roughly at random and added new downloaders for them. If you have Nitter subs, please move their 'sources' over, and they should start working again (they might need to do a bit of 'resync' and will complain about file limits being hit since the URLs are different, but give them time). If you would rather use another mirror, feel free to duplicate your own downloaders as well. Thanks to a user who helped here with some fixed-up parsers.

I gave the recently borked grouped 'status' sort in thread watchers and downloader pages another go, and I improved the reporting there overall. The 'working' status shouldn't flicker on and off as much, there is a new 'pending' status for downloaders waiting for a work slot, and the 'file status' icon column in galleries now shows the 'stop' symbol when files are all done.

The menu entry to 'open similar-looking files' is now further up on thumbnails' 'open' submenus.

The duplicate filter has its navigation buttons on the right-hand hover window rearranged a bit. It is silly to have both 'previous' and 'next' when there are only two files, so I merged them. You can also set 'view next' as a separate shortcut for the duplicate filter, if you want to map 'flip file' to something else just for the filter.

windows clean install

If you use the Windows installer, do not worry, these issues are fixed automatically for you from now on.

I updated to a new dev machine this week. Some libraries were updated, and there is now a dll conflict, where a dll from an older version is interfering with a new one. As it happens, the library that fails to load is one I made optional this week, so it doesn't seem to actually stop you from booting the client, but it will stop you from running the Client API in https if you never did it before (the library does ssl certificate generation).

It is good to be clean, so if you extract the Windows release, you may want to follow this guide this week: https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#clean_installs

Remember: Don't delete your db folder.
Open

R: 1 / I: 0 / P: 4

e621 downloader doesn't grab "lore" tags

they added a new "lore" tag section, for tags that are all on my blacklist. incest, tranny, """"""nonbinary"""""", etc.

since they're apparently separate from the other tags, my blacklist on hydrus isn't catching them.

this is a bug report, i guess. i can't use github bc they don't accept cock.li emails, can't use discord bc they require a phone number. hope you see this
Open

R: 4 / I: 0 / P: 4

Version 427

https://youtube.com/watch?v=E97ftpTjeC4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v427/Hydrus.Network.427.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v427/Hydrus.Network.427.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v427/Hydrus.Network.427.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v427/Hydrus.Network.427.-.Linux.-.Executable.tar.gz

I had an ok week. I mostly fixed and cleaned things.

more ghosts

There was another 'ghost' pending tag issue. Sometimes, on committing pending tags with siblings up to the PTR, the pending record and its count was not clearing properly. I reproduced the problem and fixed it, and a new test ensures it will not happen again. Thank you to the reports about this, they helped me to figure it out.

Since we have had some trouble with pending tags in particular, I have written a new maintenance routine to recalculate pending tags and their counts efficiently without having to wait for siblings/parents re-sync. This will run on update, so with luck, your tags and autocompletes should be fixed up (until the next problem!).

the rest

There is a new shortcut set under file->shortcuts to handle tag autocomplete shortcuts. Any 'switch searching immediately' that was previously on 'main gui' will be migrated over to this on update, and I have also moved several previously hardcoded and undocumented shortcuts to the new shortcut system, so you can now edit them! The new actions (with their defaults) are: forcing tag autocomplete lookup now (ctrl+space), flipping an IME-friendly input mode on/off (insert), navigating autocomplete results tabs left/right when input is empty (left/right arrow), navigating the service pages in manage tags dialog left/right when input is empty (up/down arrow), and navigating a parent media viewer previous/next when input is empty (page up/down).

I fixed a bad auto-setup link in the gelbooru downloader last week that messed up some users. Also rolled in is a new URL definition that should fix 420chan thread watching for their newer url format.

full list

- ghost pending tags:
- fixed another ghost pending tags bug. this may have been new or there since the display cache started, I am not sure, but it shouldn't happen again. it was occuring when a pending tag was being committed to 'current' and another tag in its sibling group already existed as a current tag for that file. the pending tag and its count would not clear for non-'all known files' domains, causing ghosts to appear in search pages but not typically manage tags. you may have noticed ghost tags hanging around after a pending commit--this was it. the true problem here was in the 'rescind pending' action that occurs just before an add/commit. a new unit test tests for this situation both for two non-ideal tags being pend-merged, and a non-ideal tag being pend-merged into the existing ideal
- wrote a routine to regenerate _pending_ tag storage and autocomplete counts from scratch for the combined and specific display tag caches. this job is special in that it regens tags instantly without having to reset sibling/parent sync. you can run the job from the database->regen menu. this is the start of 'fix just this tag' maintenance ability
- the pending regen routines will occur on update. it shouldn't take long at all, unless you have five million tags pending, where it could be a couple minutes

R: 1 / I: 0 / P: 4

Is it me or is this UI text ant-sized? I've been using hydrus on ubuntu and got used to bigger windows but now im on w10 and this is some small print, so small I dont know how I first got used to it and almost think I've done something wrong. I'm just on 1920x1080. Can I do something about this?
Open

R: 4 / I: 0 / P: 4

Version 426

https://youtube.com/watch?v=SpXsBNBtqP0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v426/Hydrus.Network.426.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v426/Hydrus.Network.426.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v426/Hydrus.Network.426.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v426/Hydrus.Network.426.-.Linux.-.Executable.tar.gz

I had a great week. I mostly fixed and cleaned things.

8kun board deleted

The 8kun /hydrus/ board is now deleted. I was planning last week to move to Endchan as the new primary, but Codexx of 8chan.moe kindly offered to host me on 8chan.moe /t/, in a new Hydrus Network General. The thread is here https://8chan.moe/t/res/2219.html . The old /hydrus/ board is also migrated to 8chan.moe /hydrus/, if you would like to search it or recover something important to you. All the links across the program and help are updated to reflect the changes. I appreciate the help from Codexx in getting this going. I have never been a comfortable or great BO, so this arrangement is a better fit for my situation for the primary place to talk Anon about Hydrus. Endchan will remain as a bunker, and I will keep making release posts here. Please feel free to use whichever you are most comfortable with.

downloaders from the future

It seems gelbooru changed their markup recently, and our default downloader stopped pulling tags, at least for some. A user helpfully created an update--which is rolled into today's release if you missed it--but unfortunately users who were running older clients ran into an unusual versioning bug. The updated downloader would only work correctly on above v422 or so, leading older clients to get a variety of annoying errors about 'md5', 'hex' or 'additional_info' when either booting or trying to inspect the broken object.

I have fixed the problem here. Anyone who recently imported a 'downloader from the future' should have it fixed today. Furthermore, the various ways you import downloaders now try to check against versions your current client can support, and if they are too advanced, you now get a nice non-spammy popup message and the too-complex objects are discarded from the import. If the advanced object is buried deeper inside the downloader, it may not be able to discard neatly yet, and you will just get a 'soft' popup message about it, but I hope to tighten this up in future.

This version checking is also applied more softly across the whole program. It is less likely that one of your internal objects will be from the 'future', but the client will now make a popup if this is so. Please let me know if you get a whole bunch of these. With luck, this is ultimately a rare problem and I can toughen up this error and actually stop clients from running if their objects ever turn invalid.

I also fixed up error recovery in the downloader system UI and multi-column lists across the program. When some borked object enters them, they now try to show an appropriate 'cannot render' style text, rather than raising an error.
Open

R: 10 / I: 0 / P: 4

Version 425

https://youtube.com/watch?v=r1nn-tp26KE [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.Linux.-.Executable.tar.gz

I had a good week. I optimised and fixed several core systems.

faster

I messed up last week with one autocomplete query, and as a result, when searching the PTR in 'all known files', which typically happens in the 'manage tags' dialog, all queries had 2-6 seconds lag! I figured out what went wrong, and now autocomplete should be working fast everywhere. My test situation went from 2.5 seconds to 58ms! Sorry for the trouble here, this was driving me nuts as well.

I also worked on tag processing. Thank you to the users who have sent in profiles and other info since the display cache came in. A great deal of overhead and inefficient is reduced, so tag processing should be faster for almost all situations.

The 'system:number of tags' query now has much better cancelability. It still wasn't great last week, so I gave it another go. If you do a bare 'system:num tags > 4' or something and it is taking ages, stopping or changing the search should now just take a couple seconds. It also won't blat your memory as much, if you go really big.

And lastly, the 'session' and 'bandwidth' objects in the network engine, formerly monolithic and sometimes laggy objects, are now broken into smaller pieces. When you get new cookies or some bandwidth is used, only the small piece that is changed now needs to be synced to the database. This is basically the same as the subscription breakup last year, but behind the scenes. It reduces some db activity and UI lag on older and network-heavy clients.

better

I have fixed more instances of 'ghost' tags, where committing certain pending tags, usually in combination with others that shared a sibling/parent implication, could still leave a 'pending' tag behind. This reasons behind it were quite complicated, but I managed to replicate the bug and fixed every instance I could find. Please let me know if you find any more instances of this behaviour.

While the display cache is working ok now, and with decent speed, some larger and more active clients will still have some ghost tags and inaccurate autocomplete counts hanging around. You won't notice or care about a count of 1,234,567 vs 1,234,588, but in some cases these will be very annoying. The only simple fixes available at the moment are the nuclear 'regen' jobs under the 'database' menu, which isn't good enough. I have planned maintenance routines for regenerating just for particular files and tags, and I want these to be easy to fire off, just from right-click menu, so if you have something wrong staring at you on some favourite files or tags, please hang in there, fixes will come.

R: 1 / I: 0 / P: 4

No discord hydrus shenanigans for now.

Locked out of discord because of their shitty mobile verification, so doing my part tagging paywalled furry art ยฏ\_(ใƒ„)_/ยฏ

> https://youtube.com/watch?v=SMTz9nIUkGc [Embed]
Open

R: 18 / I: 1 / P: 5

Version 388

https://youtube.com/watch?v=X-vdjur459c [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v388/Hydrus.Network.388.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v388/Hydrus.Network.388.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v388/Hydrus.Network.388.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v388/Hydrus.Network.388.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v388.tar.gz

I had a great week. The client can now save and load searches.

favourite searches

Every tag autocomplete input text box that searches for files--the most obvious being the one on normal search pages--now has a star icon button beside it. Click this, and you get a menu to save your current search, manage your saved searches, or load up one that is saved!

Currently, the saved information is the list of search terms (the tags and system predicates), the current file and tag domains (e.g. my files/all known tags), whether the system is 'searching immediately' or waiting, and optionally the current sort and collect.

Each saved search has a name and optionally a folder, for easy grouping. I have started all clients with an example inbox processing search, just as an example.

If you, like me, keep ten or fifteen empty search pages open (for me it is mostly different creator+inbox searches), I hope this system lets you collapse it down, making for a lighter and simpler session. It should also help for more unusual workflows like duplicate filtering and even file maintenance jobs.

There may be a couple of bugs in the system, something like a collect-by not being saved or updated correctly. Let me know how you get on!
Open

R: 6 / I: 1 / P: 5

Version 424

https://youtube.com/watch?v=YnU_j_ZA-tc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.Linux.-.Executable.tar.gz

I had a good week. There are some quality of life improvements and faster tag search across the board.

The update will take some time this week to update a cache. If you do not sync to the PTR, it will be just a few seconds. If you sync to the PTR, expect about 5-15 minutes.

faster tag search

In the second half of 2020, I tried several times to tune the database for different sorts of wildcard tag search, which is used in all autocomplete lookups and many file searches. I was sometimes able to get small clients always running well, or complicated large systems running, well, but I failed to get it good for all situations with code alone--the structure of the database tag lookup cache made the tuning difficult.

So, I have updated how that cache works. Rather than always searching one big master table, the client can now 'zoom' in on the appropriate search context based on the type of search page or manage tags dialog or whatever.

Pretty much anything related to autocomplete and tag-based file searches is faster. Most importantly, the worst-case time for these searches is greatly improved. Complicated searches, like a 'namespace:*anything*' file search, should no longer have sudden gigantic lag spikes. These searches may still take ten seconds or more when searching millions of tags and files, but they won't accidentally lag out for two minutes on some tiny 'my tags' search with only 60 results.

The only exception in my testing is 'number of tags' searches still have bad cancelability. It is better, but not great. I'll keep working here.

The cache replaces an existing one. It will take some time to build it on update. If you do not sync with the PTR, it should just be a few seconds. If you sync with the PTR on an SSD, it should be 5-15 minutes (on my heavy client with a nice SSD, it was 7 minutes). If you sync with the PTR on an HDD, it will take significantly longer, so please plan for it. If you sync with the PTR, you will see some numbers count up as it builds the different parts of the cache. There will be some deletion work to start, then counting up to perhaps a million, and then up to 16 million or so, at about 30,000 a second.

I have more plans here, and more work to do to optimise the tag display system, but I will let this new cache breathe for a bit before going back in here with a machete.
Open

R: 4 / I: 1 / P: 5

Version 423

https://youtube.com/watch?v=SvsHVu3xt6A [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v423/Hydrus.Network.423.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v423/Hydrus.Network.423.-.Linux.-.Executable.tar.gz
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v423/Hydrus.Network.423.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v423/Hydrus.Network.423.-.Linux.-.Executable.tar.gz

๐•ธ๐–Š๐–—๐–—๐–ž ๐•ฎ๐–๐–—๐–Ž๐–˜๐–™๐–’๐–†๐–˜!

I had a good week making some small fixes and improvements to finish up the year. This is the last release of the year. There is a large poll on what 'big thing' to work on next:

poll

Here is the poll on what large work to go for next:

https://www.survey-maker.com/poll3310902xA574481e-102

You can vote on multiple items. Please don't worry about the seriousness of it too much--I have a good idea of what is likely to win already, and if there are obviously jank votes, I'll reserve the right to discount a result--but I am particularly interested to know what is and is not popular further down the list. I'll take the results on the 6th of January.

The end of 2020 has come quick for me. I still have the network updates 'big job' to do, so that's first for 2021 Q1, but after that, I will plan out and hack away at the top item(s) on the poll.

If you were not aware, a team of users is doing great work managing the Github issue tracker for hydrus. There is also a process there for bumping issues with reaction votes, viewable for big jobs like so:

https://github.com/hydrusnetwork/hydrus/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc+milestone%3A%22Major+jobs%22

I regret that am doing pretty terribly at consuming my Github work queue, but we'll see how 2021 goes. I'd like to start relying on that more to better prioritise my work.
Open

R: 5 / I: 0 / P: 5

Version 422

https://youtube.com/watch?v=AHBJ68icJ_4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v422/Hydrus.Network.422.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v422/Hydrus.Network.422.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v422/Hydrus.Network.422.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v422/Hydrus.Network.422.-.Linux.-.Executable.tar.gz

๐ŸŽ‰๐ŸŽ‰ It was hydrus's birthday this week! ๐ŸŽ‰๐ŸŽ‰

I had a great week. I mostly fixed bugs and improved quality of life.

tags

It looks like when I optimised tag autocomplete around v419, I accidentally broke the advanced 'character:*'-style lookups (which you can enable under tags->manage tag display and search. I regret this is not the first time these clever queries have been broken by accident. I have fixed them this week and added several sets of unit tests to ensure I do not repeat this mistake.

These expansive searches should also work faster, cancel faster, and there are a few new neat cache optimisations to check when an expensive search's results for 'char' or 'character:' can quickly provide results for a later 'character:samus'. Overall, these queries should be a bit better all around. Let me know if you have any more trouble.

The single-tag right-click menu now always shows sibling and parent data, and for all services. Each service stacks siblings/parents into tall submenus, but the tall menu feels better to me than nested, so we'll see how that works out IRL. You can click any sibling or parent to copy to clipboard, so I have retired the 'copy' menu's older and simpler 'siblings' submenu.
Open

R: 5 / I: 0 / P: 5

Version 421

https://youtube.com/watch?v=m9r7VRNUyR8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v421/Hydrus.Network.421.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v421/Hydrus.Network.421.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v421/Hydrus.Network.421.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v421/Hydrus.Network.421.-.Linux.-.Executable.tar.gz

I had a good week. I fixed some important bugs, and cleaned up some core access and maintenance code. This week's changelog gets pretty technical, which you can safely ignore. Database works betterโ„ข.

highlights

I fixed a problem importing files with 'only add tags that already exist' checked in tag import options. Sorry, this was a stupid typo. I added a unit test here to ensure it doesn't happen again.

When you search for potential duplicates from the duplicates page, this now happens in a non-interrupting popup in the bottom-right. You can keep searching and browsing while it works. All duplicate pages sync with each other better, too, and share more CPU work.

Thanks to a user's contribution and help, the 'discord drag and drop' BUGFIX mode, which is really a 'export files to a temp location before starting the drag and drop' mode, now supports the same 'filename pattern' you see in the export files dialog. Check it under options->gui. This is experimental, but the first step to automatic export with nice filenames. If you care about export filenames, let me know how it works for you, and what sorts of filenames you want to make in future. I really want to update the old 'filename pattern' system to something with conditional 'if' statements.

If you are a new user and had trouble syncing to the PTR in the past week with an error about an invalid tag, you got caught by my new 'invalid tag catching' code being overzealous. This is now fixed, so if your PTR automatically paused due to the error, please hit services->review services->PTR and then unpause it. When a repository gives an invalid tag (we think there are about 17 really broken tags in the PTR atm, from older days), the client now recovers.
Open

R: 4 / I: 0 / P: 5

Version 420

https://youtube.com/watch?v=gUTOl_KGLEk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v420/Hydrus.Network.420.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v420/Hydrus.Network.420.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v420/Hydrus.Network.420.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v420/Hydrus.Network.420.-.Linux.-.Executable.tar.gz

I had a great week fixing a whole bunch of bugs.

bugs

I fixed taglist drag-select, which was not moving the to-be selected indices down with the scroll. Sorry for the trouble here. You can now also ctrl+drag-select to deselect.

There was a bug in the new virtual siblings and parents lookup system that meant some grandparents and siblings were not appearing. For instance, for parents, 'samus aran' might have 'metroid', and 'metroid' would have 'nintendo', but 'samus aran' would not have 'nintendo'. Thanks to help from users, I was able to reproduce it and fix the problem. When you update, the client will spend a few seconds regenerating the lookups and finding the missing links. It will queue up a bit more work for the background display sync to do later on. In my test situation, the PTR went from 189,000 sync rows to 192,000.

Autocomplete results in manage tags now list parents beneath every tag they apply to. Previously, parents could only exist in the list once, so they were accidentally de-duping and only ending up beneath the last applicable tag. Now you get plenty.

Also, these 'write' autocomplete results now show sibling and parent data for the tag that matches your input text even if that tag has no count. When that tag has sibling data, all the siblings are loaded as well. This sounds obscure, but you'll notice when you next start entering tags on a service with a lot of siblings. It should make it easier to quick-type complicated tags in manage tags.
Open

R: 8 / I: 0 / P: 5

Version 419

https://youtube.com/watch?v=3xX5B62ENrM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v419/Hydrus.Network.419.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v419/Hydrus.Network.419.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v419/Hydrus.Network.419.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v419/Hydrus.Network.419.-.Linux.-.Executable.tar.gz

I had a great week finishing editable system predicates and making some long-running search situations faster and snappier.

editable predicates

As a reminder, if you shift+double-click on the search terms (predicates) in your current search, you can now edit them! I also added an right-click 'edit' menu entry, if you do not like the shift+double-click.

OR and system:rating predicates are now editable. Also, 'invertible' predicates like inbox/archive or tag/-tag will now be stacked in the editable panel as buttons you can click to flip.

You can now set the default for any editable system predicate type! Little 'star' buttons beside the panels now let you set the current value as the new default.

The editable system predicate panels also remember the five most recently entered predicates for each type, and these are stacked up top as quick buttons. The UI here gets a little tall and jank, but see how you like it!

You can now shift+double-click tags in a place other than your current search predicates (e.g. the 'selection' tags in center-left) to add the selected tags as a bundled OR tag. Shift+middle-click does the same for a new search page. There are also right-click menu entries for these.

All tag lists now support drag selection, so you can click a tag, drag down three rows, and all will be selected without having to click more.
Open

R: 2 / I: 0 / P: 5

Version 418

https://youtube.com/watch?v=lkk6m14htzw [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v418/Hydrus.Network.418.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v418/Hydrus.Network.418.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v418/Hydrus.Network.418.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v418/Hydrus.Network.418.-.Linux.-.Executable.tar.gz

I was ill this week, so I didn't have as much work time as I wanted. I did get some basic system predicate editing done though, if you want to try that out!

editing system predicates

This works, but it is basic, not super polished.

If you shift+double-click on a system predicate that is in your active search (like 'system:filesize>200KB'), it should now launch a dialog to edit it in place. You can also edit several at once. This should make doing several 'time imported < 7 days ago' ... '6 days ago' ... '5 days ago' searches in a row to chase something down a bit easier!

If a predicate is too simple for a panel but has an 'invert' version (e.g. archive/inbox, has audio/no audio), then it flips to that invert on a shift+double-click. This also works for regular tags, flipping between 'tag' and '-tag'.

As I was short on time, I did not finish everything and could not add too many bells and whistles. The stacked edit panel can look a little ugly, there is no menu command to edit, selection disappears after an edit, I have no edit support for 'system:rating' or 'OR' predicates, and I was unable to reintroduce optional default values for system predicates. I'll go at this work next week.

In any case, give it a go and let me know what works well and what gives you trouble. I am not super enthused about shift+double-click as the edit shortcut, but it is a good compromise. I'll likely end up adding it to the shortcut system so you can edit it.

BTW: A couple of clever system pred options that were accidentally hidden are reintroduced to the new options->search page today.
Open

R: 3 / I: 0 / P: 5

Version 417

https://youtube.com/watch?v=Vim61JtH3fA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v417/Hydrus.Network.417.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v417/Hydrus.Network.417.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v417/Hydrus.Network.417.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v417/Hydrus.Network.417.-.Linux.-.Executable.tar.gz

I had a great week working on a variety of small jobs.

The hydrus network version updates this week, so any clients that currently sync with the PTR will need to update to continue syncing. There is no rush for this.

all misc this week

First of all, the hydrus network version increments from 18 to 19 this week. Clients and servers can only talk to each other if they are running on the same version, and the PTR will update today, so if you want to keep syncing with the PTR, you will have to update. This version update was to ensure that all clients syncing with tag repositories are now on the new virtual sibling and parents systems. There is no rush to get this done, and no penalty if you delay--you will just get a polite popup saying there is a version mismatch, and your PTR service will be paused.

If you get a popup about the network version mismatch in an older version, after you update to 417, go to services->review services and make sure the PTR service panel is unpaused. You might also want to hit 'refresh account'.

I added some shortcut actions to the 'main window' shortcut set--one for 'run all export folders now', and several for navigating the current page selection. You can navigate left, right, and to the leftmost or rightmost tab of the current row. Unlike (shift+)ctrl+tab behaviour, the left/right actions jump through page of pages when you are at a boundary, letting you step through all of your tabs in a depth-first manner.
Open

R: 3 / I: 0 / P: 5

Version 416

https://youtube.com/watch?v=IPjvDE-rKo0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v416/Hydrus.Network.416.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v416/Hydrus.Network.416.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v416/Hydrus.Network.416.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v416/Hydrus.Network.416.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v416.tar.gz

I had a great week catching up on a mix of smaller work.

highlights

The manage tags dialog is now more efficient at making changes for very large numbers of files. My test application of 6 tags to 10,000 files went from 52 seconds to 4.8! A variety of other tag presentation updates should also benefit from the optimisations here.

I reworked my hacky multi-list sizing code. The various instances of the final column sizing weird should be much better. The horizontal scrollbar is enabled again.

A new checkbox under options->files and trash now lets you prefix any copied hash with the hash type, like "md5:2496dabcbd69e3c56a5d8caabb7acde5". If you often paste hashes into boorus or other search engines that take this format, this should make it a bit quicker!

I am rolling in default thread watchers for warosu, prolikewoah, and smuglo.li. Warosu seems to have some CloudFlare stuff going on, so your mileage may vary.

full list

- misc:
- the new siblings and parents taglist menus now copy just the actual tag when you click, excluding the 'ideal/child/parent:' prefixes
- added a checkbox to _options->files and trash_ that allows you to automatically prefix hashes copied to clipboard with their hash type in a booru-lookup friendly manner, such as "md5:2496dabcbd69e3c56a5d8caabb7acde5"
- the media viewer now remembers if it was previously maximised when you set it to un-fullscreen (before, it would always restore-window-ise)
- fixed the 'test address' button in _manage services_ for hydrus administration services
- improved the 'add upnp mapping' error handling to better catch 'already mapped' error, with separate errors for redundant, already-on-but-wrong-port, and already-on-another-computer
- improved error handling when saving objects to the database, particularly for encoding or giganto-size-session errors
- rewrote my tag sibling lookup unit tests to deal with more situations
- wrote similar fairly comprehensive tag parent lookup unit tests
- .
- new downloaders:
- rolling in a user-created thread watcher for warosu. it may be CloudFlare hampered depending on your situation
- rolling in a prolikewoah thread watcher
- rolling in a smuglo.li thread watcher
Open

R: 2 / I: 0 / P: 6

Version 415

https://youtube.com/watch?v=SW9aJkqrUKw [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v415/Hydrus.Network.415.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v415/Hydrus.Network.415.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v415/Hydrus.Network.415.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v415/Hydrus.Network.415.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v415.tar.gz

I had a mixed week. I greatly reduced the lag that 414 could get after large sibling/parent changes, but unfortunately I could not get much else done.

all misc this week

I had wanted to do a bunch of small work this week, but my schedule fell apart a bit, and then the 'one last thing' for the new parents cache, which was rejiggering the background maintenance code to do smaller jobs at once, ended up being very difficult. I worked hard on figuring out a method, and then when I was ready to move on, I realised there was a design flaw and had to re-do a large part of it! In any case, it now works much better--even if you have thousands of tags with one common parent, they can now calculate one at a time, nice and quietly in the background. Some million-strong siblings and parents might still be a bit too laggy for an HDD to calculate in normal time, but an SSD should be generally ok. If it still lags for you, you can make it only work in idle time under tags->siblings/parents sync.

In that vein, the new tag display dialogs now have a nice green/red text label to describe the current maintenance options, so you better know when to expect any changes to be calculated.

I fixed and improved a couple small things, but other than that, the main highlight is options->gui pages now lets you put the main window's page tabs on top/left/right/bottom (previously it could only do top/left). This now also updates as soon as the options dialog is applied.
Open

R: 5 / I: 0 / P: 6

Version 414

https://youtube.com/watch?v=u3gfgq5UnNc [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v414/Hydrus.Network.414.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v414/Hydrus.Network.414.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v414/Hydrus.Network.414.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v414/Hydrus.Network.414.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v414.tar.gz

This is the first release of a complicated new system. If you are a cautious user, feel free to wait a week for me to iron out any surprise bugs.

I had a good four weeks converting tag parents to the new 'virtual' database cache.

The release does not need to do a big update this week. It will ask you if you have an SSD or an HDD (EDIT: with a special caveat for PTR users, see below), but that is it. All the mandatory heavy work that came in with 407 now happens in the background.

parents

tl;dr: You don't have to do anything. If you haven't heard of a tag parent before, no worries. The database should work better now.

I did the same to parents as I recently did with siblings. They are now virtual, which means when the database needs to 'fill in' a missing tag because of a parent relationship, that new tag now does not actually exist in storage, only in a special computed database cache. If parents change, the implications in this cache can be recalculated with no permenant changes made. Parents are now undoable.

Parents are also comprehensive. Previously, due to different logical holes, not every parent was filled in correctly. There were several retroactive holes. Parents now always apply to all their children, and they adjust to siblings and sibling changes.

Also, you can now apply parents across services. This matters less than siblings, but if you have particular parent needs, you can now set your 'my tags' parents to work on the PTR, or remove all parents entirely, under the new tags->manage where tag siblings and parents apply dialog.

This took a bunch of work. I am happy with all of it except how parents now display in manage tags. It was complicated to insert 'ghost' tags to represent parent implications and I just ran out of time, so for now, tags with parents just get a '(2 parents)' suffix after their tag label. You can now right-click on tags in most places to see all their sibling and parent data.
Open

R: 6 / I: 0 / P: 6

Version 413

https://youtube.com/watch?v=neSjixsTTjY [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v413/Hydrus.Network.413.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v413/Hydrus.Network.413.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v413/Hydrus.Network.413.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v413/Hydrus.Network.413.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v413.tar.gz

I had an ok week. I mostly fixed bugs to make a 'clean' release before I go for the tag parents database cache.

It is all misc work this week, no highlights.

full list

- added 'sort by number of files in collection' file sort type. it obviously only does anything interesting if you are collecting by something
- when you enter a tag from a manage tags suggested tags column with a double-click, the tag input box is now immediately focused. entering it with a keyboard action does not move the focus
- wrote a new routine for the 'check and repair' database menu that scans for and fixes invalid tags. this might be some system:tag that snuck in, superfluous unicode whitespace, or some weird website encoding that results in null characters, or any other old tag that has since become invalid. tag translations are written to the log
- added an experimental 'post_index' CONTEXT VARIABLE to subsidiary page parsers--whenever a non-vetoed post has pursuable URLs, this value is incremented by one. this is an attempt to generate a # 0,1,2,3 series. feedback on this would be appreciated, so I can formalise and document it
- added 'no_proxy' option for the options->connection page. it uses comma-separated host/domains, just like for curl or the NO_PROXY environment variable. it defaults to 127.0.0.1. in future, options will be added to auto-inherit proxy info from environment variables
- fixed an error when subscriptions try to publish to a page name when a 'page of pages' already has that name
- activated some old 'clean url' parsing tech I wrote but never plugged in that helps parsing urls from source fields on sites that start with non-url gubbins
- fixed the v411->v412 update step to account for a tags table that has duplicate entries (this shouldn't ever happen, but it seems some legacy bug or storage conversion indicent may have caused this for some users). if a unique constraint error is raised, the update step now gives a little message box and does dedupe work
- fixed an issue where the 'will display as' tag was rendering without namespace when 'hide namespace in normal views' was on
- fixed a recent character encoding routine that was supposed to filter out null characters
- fixed some UPnP error reporting
- _may_ have fixed an odd and seemingly rare 'paintevent' issue when expanding the popup toaster from collapsed state--it may also have been a qt bug, and fixed in the new qt:
- updated qt to 5.15.1 for windows and linux builds. it fixes a couple of odd issues like 'unclicking' to select a menu item (issue #296)
- added session save to holistic ui test suite
- misc code cleanup
Open

R: 4 / I: 0 / P: 6

Version 412

https://youtube.com/watch?v=3-K94kdbTS8 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v412/Hydrus.Network.412.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v412/Hydrus.Network.412.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v412/Hydrus.Network.412.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v412/Hydrus.Network.412.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v412.tar.gz

I had a great week catching up on smaller jobs, improving search speeds, and adding a 'lite' 407->408 update mode for HDD users who sync with the PTR. There are also a couple of new applications for the Client API.

Update this week will take a few seconds to a few minutes as new database indices are created.

sibling and search speeds

Thanks to feedback from some PTR-syncing HDD users, the new siblings update code, most importantly in step 407->408, takes way too long for them - perhaps more than 24 hours. I have written a little yes/no dialog popup into the update step that talks about this and optionally activates a 'lite' mode that does not apply siblings. This still requires some basic cache copying work, but it is significantly less. If you are still on 407 or before and have been waiting to update, please give this a go and let me know how it works for you.

The 'manage where tag siblings apply' dialog now has some red text to warn about the high CPU/HDD of applying many siblings to a large number of tags. I am still not happy with the 'monolithic' way this db work goes, so when I get stuck into the parents cache, I will write an asynchronous system that does this work in the background, pause/resumable, without interrupting browsing and so on, much like I did with repository processing.

Some things were working slow since siblings (e.g. in a search, mixing wildcard tags with regular tags), but I went through every instance of my new optimisation code, fixing bugs, testing it at large scale, and smoothing spikes out further. Tag, namespace, wildcard, tag presence/count, known url, and file note searches should all be more reasonable. A neat new tag search pre-optimisation routine that checks autocomplete counts for expected result size before deciding how to search now works for more sorts of tags and also kicks in for namespace and wildcard searches, which now break their work into smaller and simpler pieces. I also added and reshaped some database indices, which will ensure that more unusual search types and general operations can still run efficiently. The update will take a few seconds to a few minutes as tag indices are regenerated.

I have learned a bunch about speeding up multi-predicate searches recently - how to get it wrong and how to get it right. I have a plan to speed up rating and known url results, which are still generally not able to speed up with multiple predicates on large clients.
Open

R: 3 / I: 0 / P: 6

Version 411

https://youtube.com/watch?v=NgcYLIvlp_k [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v411/Hydrus.Network.411.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v411/Hydrus.Network.411.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v411/Hydrus.Network.411.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v411/Hydrus.Network.411.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v411.tar.gz

I had a good week. The siblings work is complete.

siblings

The last areas that used the old siblings systems are now migrated to the new fast cache. The 'loading tag siblings' step of client boot, which for PTR users could be several seconds of work, is no longer needed!

These last parts are all user-facing UI code. Mostly tag autocomplete behaviour and the 'will display as' labels you see in tag edit contexts. Everything now goes through the database, and all in a nice asynchronous way, so (fingers crossed) it will not add noticeable lag.

As I unified code here and moved to nicer logic, I was able to clean up some behaviour. Matching siblings based on what you type (for instance, if you have 'lotr' siblinged to 'series:lord of the rings', text-matching the latter even if you type the former) should be cleaner and more reliable. Also the 'will display as' labels should appear in all the tag suggestion columns, and some other 'edit tag' places.

There is still more I could do here. Using 'manage tags' on thousands of files at once can be a bit laggy, both in working and once you hit apply, and I could make '(will display as xxx)' labels colour according to the xxx namespace to make them 'pop' a bit, and I would still like to ultimately replace the individual 'will display as' labels with something neater for workflow. But I have been focusing on siblings for more than a month. Now the main virtualisation and boot lag elimination objectives are done, I should move on to other work.

If you have been waiting to update from 407, this is a fine week to do it. There will still be a whack of update CPU if you sync to the PTR, but on an SSD it shouldn't be more than a one-time shot of 10-30 minutes.
Open

R: 3 / I: 0 / P: 6

Version 410

https://youtube.com/watch?v=N7GaWASGcgs [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v410/Hydrus.Network.410.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v410/Hydrus.Network.410.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v410/Hydrus.Network.410.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v410/Hydrus.Network.410.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v410.tar.gz

I had an ok week. I wasn't as productive as I hoped, but I am happy with the mostly optimisation work.

optimisations

After some more profiling in IRL situations, and with more helpful info from users, I have done another round of profiling for the new sibling cache, and more besides. A database technique I use for many purposes is now more reliable (fewer lag spikes), and has less CPU overhead. If you found some systems (like the 'related tags' suggestions in manage tags dialog) sometimes took a few seconds to work in the past couple of weeks, they should now be fast again. And you should find many types of file search, particularly those with multiple search predicates, and general tag processing, should be faster than before.

My dev machine went from about 3-8k rows/s processing speed in a test environment up to 8-20k rows/s, which is faster than it was before the siblings cache was added.
Open

R: 3 / I: 0 / P: 6

Version 409

https://youtube.com/watch?v=puU4EvBTPxA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v409/Hydrus.Network.409.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v409/Hydrus.Network.409.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v409/Hydrus.Network.409.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v409/Hydrus.Network.409.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v409.tar.gz

I had a great week fixing some bugs and optimising the new tag siblings cache. The new code works much faster now.

siblings

I am very happy that there do not seem to have been any obvious errors with the new sibling database cache. Unfortunately, a couple of areas were working inefficiently, which IRL testing helped to diagnose. I put a lot of time into this this week and was very successful - some sections take 10% less time, some 90%, and one critical query now takes 99% less time. It depends on many factors, but many things are faster overall. In particular, tag processing speed, which took a real hit, is back up to good speed, and setting new tag sibling application rules now only needs to regenerate for changed siblings, so if you add (or remove) your own five 'my tags' siblings onto the PTR, the client now only has to do two seconds of work, not ten minutes.

I made some progress on the final awkward things to migrate. Most autocomplete results you see are now able to give themselves the 'will display as xxx' label when needed and match against sibling input (e.g. having an input of 'lotr' match 'series:lord of the rings' due to siblings) on their own, which should save some CPU time when typing. There is still more to do, so I'll keep hammering at it for the next two weeks and see if I can get rid of 'loading tag siblings' on boot before I start on parents db cache.

If you have been waiting for faster code before you update, you might want to wait another week. I just did a test re-do of the 407->408 update step in IRL conditions, and it was not as fast as I wanted it. I'll keep pushing at this.

I am increasingly looking forward to doing that parents db cache, which will extend this work in a new dimension. That will be v412, which I am very confident will be another two-week release. This is some of the most intricate work I have done.
Open

R: 6 / I: 0 / P: 6

Version 408

https://youtube.com/watch?v=X8zLP-TBLcA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v408/Hydrus.Network.408.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v408/Hydrus.Network.408.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v408/Hydrus.Network.408.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v408/Hydrus.Network.408.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v408.tar.gz

I had a great couple of weeks moving tag siblings forward. The update this week will take some time as a new cache is generated. On an SSD with the PTR synced, about 10 to 20 minutes, depending on how many files you have. Some of the new tag sibling application rules may have 5-10 minute delays on edit as well. If you have a large/PTR client and are on an HDD, or you would rather just wait for the kinks to be ironed out, you might like to wait a week or two for me to further optimise this new code.

siblings

tl;dr: Siblings are faster and better now, you don't have to do anything. Some parents will not appear with new downloads - don't worry about it, they will all fill back in nicely soon.

I have finished the first version of the database 'display tags' cache. This is a long-planned system that keeps track of how tags are 'collapsed' by siblings on an ongoing basis. Until now, siblings were always calculated on the fly at the UI level whenever media was loaded, which lead to slow and inaccurate sibling-based search results and a variety of logical problems. Search results generally load much faster now, particularly on subsequent loads.

On the front end, tags appear exactly they did before, but when you do tag-based searches or tag autocomplete, the files and counts that come back should be more accurate. For instance, if you have a file tagged 'metroid' that appears as 'series:metroid', and you search 'series:anything', that file now shows. Siblings logic is also tighter, and changes apply faster.

And now, if you go to services->manage where tag siblings apply, you can now decide which services' siblings apply where. If you do not like the PTR's siblings, or you have a couple of different preferences you want to add on top, you can say things like 'apply siblings from "my tags" then "PTR"', or you can turn them off entirely. This powerful service-based system replaces the old 'apply all siblings to all services?' checkbox that used to be in the options.

This is some complicated code affecting core systems. I have tested it extensively, but IRL situations, particularly with large sessions or storage, will be more complicated and useful than what I have here. Please let me know where this new system is laggy or miscounts anything. And of course, if anything outright breaks, let me know!

The help on siblings is going to be increasingly outdated as this work continues. If you don't know what siblings and parents are, please wait until I am done with this multi-month job, and I'll have some nice new help written.
Open

R: 6 / I: 0 / P: 6

Version 407

https://youtube.com/watch?v=3Rup3EdA0kw [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v407/Hydrus.Network.407.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v407/Hydrus.Network.407.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v407/Hydrus.Network.407.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v407/Hydrus.Network.407.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v407.tar.gz

I had an ok week. I did some prep work for siblings, and fixed some bugs.

misc

The right-click menu off a normal taglist now provides a whole bunch of different copy options, depending on whether you want all or selected, tags or subtags, and with or without counts.

I fixed a bunch of weird hover window issues in the media viewer. On a thin media viewer, the top-right window now squeezes into the smaller space better, and if it can't fit, it bumps down so you can still use it. The center-right duplicates window also positions correct on a thin window. And now moving your mouse off a clicked hover window will immediately restore focus to the main media window, which should eliminate some annoying click focus issues (usually when you click quickly after moving your mouse off, when the click can be swallowed).

A user has written a simpler getting started guide here: https://github.com/Zweibach/text/blob/master/Hydrus/Hydrus%20Help%20Docs/00_tableOfContents.md This is now linked at the top of my help as well. If you have had trouble with the denser official help, or you know someone else interested in hydrus who is similar, please give that a go.
Open

R: 2 / I: 0 / P: 6

Version 406

https://youtube.com/watch?v=UZr6b4mBoxI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v406/Hydrus.Network.406.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v406.tar.gz

I had a good week. It is now easy to deduplicate queries in subscriptions.

subscriptions

Managing multiple subscriptions can get complicated. Figuring out which queries are in which sub, whether and where you have duplicates, is tedious. A recent bug with query pasting also could have introduced some duplicates within the same sub. This week should make it simple to manage.

The main manage subscriptions dialog now has a 'deduplicate' button. It lights up any time your subs of a particular downloader have the same query, whether that includes dupes within the same sub or across different subs. Click it, and it launches a thorough process to determine what you want to do:

- First, it asks if you want to deduplicate by case or not--e.g. whether "Samus_Aran" and "samus_aran" should be considered dupes, or only exact matches.
- Then it summarises which downloaders appear to have dupes, letting you choose which to work on.
- Then you have the option of selecting which of the queries you wish to deduplicate--e.g. if you want to do two runs to dedupe, say, artists to one sub and characters to another.
- Then you select which subscription(s) will retain the deduplicated queries.

Every step explains itself. Mostly it is simple, but there are some odd situations. When a sub dedupes queries within itself, it keeps the one(s) with the most files. I hope I have accounted for everything reasonable.

Adding to caseless matching, also added is a 'lowercase' button, which will coerce all the queries in the selected subscriptions to lowercase, just to help keep things neat. And lastly, when you paste queries into a sub, the 'already in the sub' test is now caseless.
Open

R: 3 / I: 0 / P: 7

Version 405

https://youtube.com/watch?v=6e5yyIcq40o [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v405/Hydrus.Network.405.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v405.tar.gz

I had a good week. 'system:number of tags' now supports namespace filtering.

number of tags

The 'system:number of tags' predicate now lets you attach a namespace, so you can search for 'files with two character tags' or 'files with more than one creator', or any other combination you can think of. Also, all 'number of tags' queries have been optimised, and are now typically much faster, particularly when mixed with other tags. They are also quickly cancellable, so if you do enter a very slow one, it should respond fairly quickly if you hit the 'stop search' button that appears after a delay.

I updated a bunch of the database code here to support new types of search. With the namespace searching, you can enter a blank namespace to search for 'unnamespaced' tags, and if you have a very specific need, you can now enter a wildcard namespace, like 'crea*', perhaps to help search for typos.

Unfortunately, I am removing 'system:number of tags' from the 'default system predicates' options panel. EDIT: I am hiding the whole panel now. This horrid stack of UI runs on ancient and difficult-to-update code, so rather than wrestle with it more, I plan to retire it and replace it with a clean system that also allows for editing search predicates in place (e.g. right-clicking an active search predicate and changing the >2 to >3 or similar).

I am really happy with how namespace counting turned out, and I have wanted it myself for a long time. Let me know how it works for you.

duplicate subscription queries

I believe the subscription data storage rewrite introduced a bug where when you paste queries into a sub and it says 'x y z are already in the sub and will not be added', they were being added anyway!

I fixed it. If you paste into subs a lot and get this message regularly, please check your subs for dupes! Now we have faster and more flexible subs, some users have also asked for broader de-duping tech that spans across multiple subs or does upper/lower case deduping, so I also expect to write buttons to do this in the near future, so you may also just want to wait for that.
Open

R: 3 / I: 0 / P: 7

Version 404

https://youtube.com/watch?v=J84FPJAb7rM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v404/Hydrus.Network.404.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v404/Hydrus.Network.404.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v404/Hydrus.Network.404.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v404/Hydrus.Network.404.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v404.tar.gz

I had a good week working almost entirely on the client's multi-column lists. They now remember their column widths and sort.

multi-column lists

All the tables across the program are now plugged into a unified options structure. This remembers the lists' columns' widths, so if you resize the manage subscriptions list a little tighter, then the next time you load that dialog, it should come back the same. The lists across the program that automatically start and maintain a sort (e.g. manage subs, or the query list in a downloader page), also remember if you change the sort.

This sounds simple, but it involved rewriting critical sections for every list, and there are about 66 different ones! I am happy with the change, as it means a load of duct-tape mess is unified and easier to maintain and extend in future.

I built the system with several near-term extensions in mind: having all lists sort cleanly, quickly, and by default, and show sort up/down arrows; allowing columns to be rearranged; and allowing columns to be hidden, which will also allow me to add complicated columns (e.g. statistically expensive subscription quality info) that start hidden by default.

Although the system is now easier to check for errors, the sheer number of lists and amount of code I changed this week means there is probably a typo somewhere in a column name or initial size. I have checked the main lists closely, but if you encounter anything odd, or even a list that does not boot at all, please let me know. There are still a couple of sizing calculation peculiarities I need to fix.
Open

R: 6 / I: 0 / P: 7

Version 403

https://youtube.com/watch?v=iMzP0ZK-J7U [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v403/Hydrus.Network.403.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v403/Hydrus.Network.403.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v403/Hydrus.Network.403.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v403/Hydrus.Network.403.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v403.tar.gz

Despite losing a couple days to sickness, I had a great two weeks doing small jobs.

shortcuts

I reworked some of the behind the scenes of shortcut commands, most importantly cleaning up a bad decision I made when I first moved to the new system. There are about 120 'simple' commands in the system, and now all of them are defined in a neater way that allows me to change and expand things more easily in future.

Every shortcut now has a nicer 'label' when you select it in the edit shortcuts UI (and anywhere else the command is shown). Rather than the old 'focus_media_viewer', you'll get something like 'keyboard focus: to the media viewer'. Rating shortcuts will also display their service's rating as it understands it, like 'set to 3/5' instead of the ugly internal data of 'set to 0.6'.

Although I have been wanting to add more actions to the system--mpv commands have been delayed far longer than I wanted--I have been putting that off until this was done. I hope to push on this more in the near future.
Open

R: 12 / I: 2 / P: 7

Version 402

https://youtube.com/watch?v=e2VTFTFufHk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v402/Hydrus.Network.402.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v402/Hydrus.Network.402.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v402/Hydrus.Network.402.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v402/Hydrus.Network.402.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v402.tar.gz

I had a great week just doing cleanup and other small work.

all misc this week

Most of the pages in the client now have special status bar text for when they have no files. If your search results in no files, instead of the old '0 files', you'll get 'no results for this search', and 'search cancelled!' if the search was cancelled, and so on. It just adds a bit of quick feedback to some actions--like highlighting a gallery query with no new files--where the new blank page may just be replacing another blank page.

I added 'copy_xxx_hash' shortcuts to the media shortcut set, for 'md5', 'sha1', and 'sha512'. All the hash-copying code is neated up as well, so if hashes can't be found for some reason, you'll still get what could be found and get nice info on how many were missing and why.

The client's shutdown code is untangled. A variety of unusual shutdown situations should be smoother, less buggy, maybe a little faster, and when errors do occur, they should be caught and reported more gracefully.

Gallery and watcher pages' list right-click menus now have direct links to the selected query or watcher's file and gallery logs. It is just an easier shortcut than highlighting and finding the button.

Forced file maintenance jobs now list a correct total x/y progress on their popups. Previously, they would show x/256 in the batches of 256, with no total summary.

Most of the lists with clipboard/png import/export buttons now also support json files, which is just a straight copy of what would go to your clipboard, but to a file. You can drag and drop compatible json files on the lists as well, just like with pngs.

I have updated the Windows and Linux client to Qt 5.15, up from 5.13. We had a variety of problems with 5.14, but this version seems to be working well. Let me know if you have any trouble, or if any problems are magically fixed.
Open

R: 4 / I: 0 / P: 7

Version 401

https://youtube.com/watch?v=uHn9WnHe2eQ [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v401/Hydrus.Network.401.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v401/Hydrus.Network.401.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v401/Hydrus.Network.401.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v401/Hydrus.Network.401.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v401.tar.gz

I had an ok week, although I was a little short on work time. This is mostly a bugfix release that tweaks how the new subscriptions work and fixes some downloader update issues. There is also an important new way to make feature suggestions for hydrus.

mostly misc work this week

Now that subscriptions can run in small efficient pieces, I removed several old limiters that stopped the old system running too often. Subscriptions should boot with less delay when needed (e.g. after the client loads or when you ok the dialog after changes), and an old system that batched queries together is gone, so if a query is due, it should run generally as soon as it can.

I also re-added the import/export/duplicate buttons to the manage subscriptions dialog. It also supports importing (and converting to the new format) the old subscription format, so if you have an old .png sub, or you want to reimport the text that was backed up to your database folder last week, this is now possible. Exporting or duplicating a very large subscription may now need a few seconds to do some database work, but the dialog will warn you beforehand.

As users have made and shared more custom downloaders, I have had reports here and there about new default downloaders sometimes messing up the custom links, unintentionally resetting back to those defaults or other unusual link remapping. I have tightened up all this code. Please let me know if you have any more trouble.

And speaking of new defaults, I am rolling out new objects for nitter, derpibooru, and e621 again this week to improve object linking and URL matching. If you had a subscription complain last week about an e621 gallery URL not matching, this should now be fixed.
Open

R: 1 / I: 0 / P: 7

github issues

It looks like Hydrusdev enabled "issues" on the github repo a few hours ago. Was there any update or notification given by him about this? He seemed pretty insistent on using the boards and his own personal to-do list for keeping track of development and bugs, so I'm curious if he explained somewhere why he changed his mind.
Open

R: 4 / I: 0 / P: 7

Version 400

https://youtube.com/watch?v=BU1F3uXyxkA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v400/Hydrus.Network.400.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v400/Hydrus.Network.400.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v400/Hydrus.Network.400.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v400/Hydrus.Network.400.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v400.tar.gz

๐ŸŽ‰๐ŸŽ‰๐ŸŽ‰ MERRY v400! ๐ŸŽ‰๐ŸŽ‰๐ŸŽ‰

I had a great week of vacation, and then a great week finally getting the subscription data overhaul done.

subscriptions

When I first wrote subscriptions, they could only hold one simple query each. Queries have become much more complicated since then, and subscriptions can of course hold many queries at once, sometimes hundreds. The old monolithic method of storing and loading subs was creaking at the seams. This week fixes it, and subscriptions should now load and operate quickly for all normal operations.

Subscriptions are now broken into pieces. Essentially, instead of one thing holding everything, they now store each query as a separate object and load and save each from your database as they are needed. The 'top' of a subscription is now always in memory and allows the manage subscriptions dialog to start instantly. Subscriptions can also boot quickly, and will cause less lag as they finish up. It all saves time and database read/write. The old '200,000 files' limit for subscriptions is gone. I wouldn't advise you make a sub with 10,000 queries just yet, but you do not have to worry about the size of any one sub too much any more.

There are no significant changes to how subscriptions look or are edited. All your existing subscriptions will be converted to the new format on update. However this is a big change behind the scenes, and if you have big subs, it may take a minute or two to update your databases. Your old subscription objects will also be backed up to a new subdirectory in your db directory, just in case anything goes wrong now or in the near future.

Unfortunately, as subscriptions are now more complicated, I did not have time to write new import/export system for them. The duplicate/import/export buttons on the manage subscriptions dialog are hidden for now.

This took a lot of planning, prep, and work. I hope you find your subscriptions work nicer, and if you have any trouble, please let me know.

new downloaders

Twitter retired their old API on the 1st of June, which broke our downloader. There is unfortunately no good hydrus solution for their new API, but thanks to a user's efforts, I am rolling in a parser for nitter, a twitter wrapper, this week. It has three downloaders--one for media posts, one for retweets, and one that does both--so please play with it and then move your twitter subscriptions over to it.

Also fixed should be derpibooru search and the md5 hash parsing of the danbooru downloader (which speeds up some downloading).
Open

R: 10 / I: 1 / P: 7

Version 399

https://youtube.com/watch?v=i_u3hpYMySk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v399/Hydrus.Network.399.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v399/Hydrus.Network.399.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v399/Hydrus.Network.399.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v399/Hydrus.Network.399.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v399.tar.gz

I had a great week tidying up smaller issues before my vacation.

all small items this week

You can now clear a file's 'viewing stats' back to zero from their right-click menus. I expect to add an edit panel here in future. Also, I fixed an issue where duplicate filters were still counting viewing time even when set in the options not to.

When I plugged the new shortcuts system's mouse code into the media viewer last week, it accidentally worked too well--even clicks were being propagated from the hover windows to the media viewer! This meant that simple hover window clicks were triggering filter actions. It is fixed, and now only keyboard shortcuts will propagate. There are also some mouse wheel propagation fixes here, so if you wheel over the taglist, it shouldn't send a wheel (i.e. previous/next media) event up once you hit the end of the list, but if you wheel over some hover window greyspace, it should.

File delete and undelete are now completely plugged into the shortcut system, with the formerly hardcoded delete key and shift+delete key moved to the 'media' shortcut set by default. Same for the media viewer's zoom_in and zoom_out and ctrl+mouse wheel, under the 'media viewer - all' set. Feel free to remap them.

The new tag autocomplete options under services->tag display and search now allow you to also search namespaces with a flat 'namespace:', no asterisk. The logic here is improved as well, with the 'ser'->'series:metroid' search type automatically assuming the 'namespace:' and 'namespace:*' options, with the checkboxes updating each other.

I fixed an issue created by the recent page layout improvements where the first page of a session load would have a preview window about twenty pixels too tall, which for some users' workflows was leading to slowly growing preview windows as they normally used and restarted the program. A related issue with pages nested inside 'page of pages' having too-short preview windows is also fixed. This issue may happen once more, but after one more restart, the client will fix the relevant option here.

If you have had some normal-looking files fail to import, with 'malformed' as the reason, but turning off the decompression bomb check allowed them, this issue is now fixed. The decomp bomb test was itself throwing an error in this case, which is now caught and ignored. I have also made the decomp bomb test more lax, and default off for new users--this thing has always caught more false positives than true, so I am now making it more an option for users who need it due to memory limitations than a safeguard for all.
Open

R: 11 / I: 2 / P: 7

Version 398

https://youtube.com/watch?v=agfhkR6JYEQ [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v398/Hydrus.Network.398.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v398.tar.gz

I had a good work week. Tag autocomplete gets some new search options, and advanced users who make downloaders get some new text processing tools.

tag autocomplete

When I recently overhauled the tag autocomplete pipeline, I eliminated some unusual logical hoops where you could accidentally fire off expensive searches that would fetch all tags. Now the code is clean, I am adding them back in as real options.

The main thing here is that services->tag display is now services->tag display and search. It has several new options to change search based on what the autocomplete's current 'tag domain' is (i.e. what the button on the dropdown says, "all known tags" or "my tags" or whatever else). The options are available for every specific tag domain and the "all known tags" domain, and only apply there.

There are three new search options: You can have full namespace lookup, so an input of 'ser' also finds 'series:metroid' and all other series tags; you can have an explicit input of 'series:*' show all 'series' tags; and you can have '*' show all tags. These queries are extremely expensive for a large service like the public tag repository (they could take minutes to complete, and eat a ton of memory and CPU), but they may be appropriate for a smaller domain like "my tags". Please feel free to play with them.

There are also a couple of clever options setting how 'write' autocompletes (the ones that add tags, like in the manage tags dialog) start up, based on the tag service of the page they are on. You can set them to start with a different file or tag domain. Most users will be happy with the defaults, which is to stick with the current tag domain and "all known files", but if you want to change that (e.g. some users like to get suggestions for "my tags" from the PTR, or they don't want tag counts from files not in "my files"), you now can. The old option under options->tags that did the "all known files" replacement for all write autocompletes is now removed.

I have optimised the database autocomplete search code to work better with '*' 'get everything' queries. In the right situation, these searches can be very fast. This logic is new, the first time I have supported it properly, so let me know if you discover any bugs.
Open

R: 9 / I: 2 / P: 7

Version 397

https://youtube.com/watch?v=2kEC7X1FUIg [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v397/Hydrus.Network.397.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v397/Hydrus.Network.397.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v397/Hydrus.Network.397.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v397/Hydrus.Network.397.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v397.tar.gz

I had an ok week. I mostly did small fixes and improvements.

highlights

I added 'system:has/has no note with name xxx' to search for files with specific note names. This is bundled into the same dialog as last week's 'number of notes', for the new generic 'system:notes' entry in the normal system predicates list.

When you have many page tabs in a row, they will now middle...elide their name text to try to fit better. You can turn this off if you like under options->gui pages.

When files in the duplicate filter are pixel duplicates, the file size and import time comparison statements now have no 'score' and colour blue. Pixel duplicates are often a different judgment call, so until I can get more flexible scoring options here, I am hardcoding it to be less visually green/red.

If you need to run a gallery download with a search term that includes '#', this should now work.

I updated the IPFS code to work with the new IPFS 5.0 daemon. I think I have things mostly correct, but the new daemon has some new CORS restrictions that may need configuration on the IPFS side, depending on your situation, to give hydrus permission to talk to it.

The left page panel's position should set itself more reliably for users who start the program minimised to system tray. If you have had trouble here, please let me know how you get on. I believe the changes here will also reduce some UI lag for many other page adding and moving operations, particularly for large sessions.
Open

R: 14 / I: 2 / P: 8

Version 396

https://youtube.com/watch?v=QM6JUHp3OZI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v396/Hydrus.Network.396.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v396/Hydrus.Network.396.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v396/Hydrus.Network.396.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v396/Hydrus.Network.396.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v396.tar.gz

I had a good week. The client now has better file 'notes' support.

There are several database updates this week. Users with large databases can expect the first boot to take a few minutes.

notes

A long time ago, I added prototype 'notes' to files. You can access it under a media right-click->manage->notes. It was a bit hacky, functioning just like a plain .txt attached to files, but some users really picked it up. This week it becomes more mature.

Firstly, files can now have multiple notes, each with their own name. Existing notes will get the simple default name of 'notes'. Also, files that have notes now show a notes icon on their thumbnails and in the media viewer top-right hover window. Clicking the hover window icon quickly opens the edit notes dialog.

And to find files with notes, there is now 'system:number of notes'. Its panel has a couple of quick buttons for finding files with any notes or no notes. Notes are stored in a faster and more space-efficient way, so this should work fairly quickly for most situations.

The next step here is to add note parsing to the downloader system. The main thought is to get artist post text and similar imported with nice 'blah gallery artist comment' style names as you download files. I had hoped to figure something out for this this week, but I didn't have time to do it properly, so I focused on polishing the above. I would like to fold parsing in in upcoming normal weekly work.

The new storage system for notes also builds a fast search cache for their text. It will not be extremely difficult to add some sort of 'system:note has text xxxxx' in future as well, if you want to search for parsed mega links or whatever else. Beyond that, I'd love some note 'preview' on the media viewer and comprehensive import/export/Client API support for notes.
Open

R: 10 / I: 0 / P: 8

Version 395

https://youtube.com/watch?v=1gl44FAOhQ0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v395/Hydrus.Network.395.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v395/Hydrus.Network.395.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v395/Hydrus.Network.395.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v395/Hydrus.Network.395.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v395.tar.gz

I had a good work week. I fixed a variety of bugs, improved the feel of the suggested tags columns, and made zooming in the media viewer nicer.

suggested tags quality of life

I recently did some work on the different 'suggested tags' panels the manage tags dialog can support. This week I worked on some of the feedback from that pass, hopefully making it a bit easier to get what you want done. As a reminder, you can set these columns up under options->tag suggestions.

The columns are now 'synced' with the current media. They all refresh/repopulate reliably when you move to another media in the media viewer. Also, all columns now hide tags that all the selected media already have, and they update this ~as you edit~, so if both 'favourites' and 'related' suggest 'character:samus aran', and you add that through any means, both lists will remove that tag! Furthermore, if you remove a tag that one of those lists would show, they now re-show that tag instantly, without having to do any new lookup.

Furthermore, the tag lists now try to retain the selection location when you add a tag through the keyboard enter key, so even though a tag you hit enter on will disappear, the focus should stay on the tag above or below. I hope this makes it easier to work with these columns with just the keyboard. As a second remind to keyboard users, you can currently set up some slightly mickey-mouse shortcuts to move focus around here under the 'main_gui' shortcuts, the actions starting 'show_and_focus_manage_tags_...'.

zoom center

When you zoom a file in or out, there is a 'centerpoint' about which the zoom happens. This position 'stays still', while everything else grows or shrinks around it. Until now, this has been hardcoded on the media center, which was sometimes unhelpful when the media had been panned. This centerpoint is now customisable under options->media. It now defaults to the media window centerpoint, which means the image will always grow and shrink around the center of what you are looking at. This makes it much nicer to pan and zoom in the duplicate filter. You can change it back to the media centerpoint, the media top-left corner, or even the current mouse cursor position, for RTS-vidya-like zooming.
Open

R: 4 / I: 0 / P: 8

Version 394

https://youtube.com/watch?v=ljExTEPNFnM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v394/Hydrus.Network.394.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v394/Hydrus.Network.394.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v394/Hydrus.Network.394.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v394/Hydrus.Network.394.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v394.tar.gz

I had an ok week mostly cleaning code.

The bug from last week where subscriptions that had incomplete file queues were unable to start is fixed. I apologise, this was a typo that was not caught by my code checker or my testing routine.

autocomplete cleanup

The tag autocomplete code has seen many updates and expansions over time, and the code had become a mess of hardcoded hooks and logical duct-tape. This week I went into it and pulled all the text parsing and results caching code out to single separate dedicated objects. Essentially most of the big changes this week are all behind the scenes, and you can check the changelog for more detailed info, but multiple small bugs and odd situations with autocomplete entry are now fixed.

A user recently contributed code that parses system predicates from text input. Now I have this nice new parsing system, I hope to integrate this new feature soon.

If you encounter new bugs or certain complicated wildcard lookups that do not work, please report them--it is now much easier for me to fix this stuff and support new search types.

environment changes

If you patch my source code, please note that the source structure has changed. The single flat directory of py files is now split into some basic subdirectories. This is one more step forward in getting the structure more manageable, and I expect to do more in future cleanup weeks.
Open

R: 14 / I: 7 / P: 8

Version 393

https://youtube.com/watch?v=Vy01sI_plzs [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v393/Hydrus.Network.393.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v393/Hydrus.Network.393.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v393/Hydrus.Network.393.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v393/Hydrus.Network.393.-.Linux.-.Executable.tar.gz

EDIT: This release had a hotfix two hours after initial release. If you got it early from github or elsewhere, please redownload. The links are the same, the build is new.

I had a good week catching up on messages and small jobs. There is also a (hopefully) neat prototype solution for some cloudflare issues.

cloudflare and network

CloudFlare hosts content for many sites online. They have a variety of anti-DDoS tech, normally a variation on a 'is this a web browser?' test, that sometimes stops hydrus from downloading. If you have seen unexplainable 503 errors on a site that works ok in your browser, this may have been it.

This week, I am adding a library, cloudscraper, to my network engine to try to solve this. Now hydrus will attempt to detect CF challenge pages when they are downloaded and pass them off to this new library, which attempts to solve the javascript challenge (the part where a CF site sometimes says 'this process may take five seconds...', just as a browser would, and then copies the solved cookies back into the hydrus network session.
Open

R: 9 / I: 4 / P: 8

Version 392

https://youtube.com/watch?v=JKPDFQRmG_M [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v392/Hydrus.Network.392.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v392/Hydrus.Network.392.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v392/Hydrus.Network.392.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v392/Hydrus.Network.392.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v392.tar.gz

I had a mixed week, but I am happy with my work. Some large database jobs are faster and tag autocomplete results have better sibling support.

tag sibling database cache

I was not able to get as much done here as I had hoped, but I was able to finish the first step and fix some sibling bugs. The client now keeps a cache for fast sibling lookups, which will be a great tool going forward.

The tag autocomplete system now uses this new cache. When you type in the start of a tag, possible matches' siblings are added to the results list, even if they do not strictly match what you typed. These siblings used to be computed on the fly, but now it is looked up using fast pre-generated data. I have also improved the accuracy of the search and added 'pending' siblings in.

As I was working here, I also fixed some sibling presentation and search bugs. If you discovered that a 'series:blah' search was delivering some unexpected 'cosplay:blah' results, this should now be fixed. There is still more to do here, which I hope to naturally fix as I replace more old systems with nice new caches.

There are two next steps here: first to write a 'what file tags should look like with siblings collapsed' cache, which will speed up tag search and file load and also eliminate many of the uncertain counts you get with autocomplete results like 'character:samus aran (500-700)'. Second is to write a full-featured 'which siblings apply to this service?' options system, so you can say 'ignore the ptr's siblings, use mine instead' and other clever combinations, rather than the current 'always apply all to all, or one to one'.

the rest

e621 and sankaku get revised default downloaders, fixed for getting 'rating' and 'genre' tags respectively.

The system tray icon got some bug fixes. It is also more stable under non-Windows, but doesn't seem to hide reliably yet.

The hydrus server and the client's API and booru now try to host their services on IPv6 as well as IPv4. This is new stuff to me, so let me know if you come across any problems!
Open

R: 7 / I: 6 / P: 8

Version 391

https://youtube.com/watch?v=l3oxjanw72w [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v391/Hydrus.Network.391.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v391/Hydrus.Network.391.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v391/Hydrus.Network.391.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v391/Hydrus.Network.391.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v391.tar.gz

I had a great week. The client now has 'minimise to system tray' support, you can search by framerate and 'number of frames', and a bunch of bugs are fixed.

system tray

This first version is buggy/crashy under non-Windows, so I have disabled it for non-advanced mode non-Windows users for now. The icon menu doesn't work well for non-Windows, but double-clicking seems ok. I will keep working on this.

The hydrus client can now make a system tray icon for those OSes that support it. You can customise it under the new options->system tray panel. You can set the client to minimise to and/or close to system tray, and you can set the icon to always display if you like. There is also a new 'hide_to_system_tray' action under the 'globals' shortcut set.

Clicking on the system tray icon will raise a 'shown' client to the front or show a hidden client. Double-clicking or middle-clicking will do hide/show. The icon also has a menu to show/hide the client, pause/unpause subscriptions or network traffic, and exit the program.

You can also set the client to boot minimised into the system tray, but there may be layout issues once you do show the client--please let me know if you get any!

frames

You can now search for file framerate or 'number of frames' under an expanded 'system:duration' search predicate. It is difficult with the data hydrus currently knows to get very precise framerate, so system:framerate searches +/-5% of what you set. You can't differentiate between 30fps and 29.97fps yet.

You can now sort by number of frames.

Video framerate is now shown in the 'file info summary' text that displays on the status bar, thumbnail menu, and media viewer.
Open

R: 11 / I: 2 / P: 8

Version 390

https://youtube.com/watch?v=HQW7I62TNOw [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v390/Hydrus.Network.390.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v390/Hydrus.Network.390.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v390/Hydrus.Network.390.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v390/Hydrus.Network.390.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v390.tar.gz

I had a good week. I spent most of my time cleaning code, but there are a couple of important bug fixes as well.

highlights

Several users have reported that potential duplicate pairs could sometimes come up again for files that had previous been set not related/false positive. This particularly hit users who had reset their potential dupes and re-searched. I had trouble reproducing this, but thanks to the latest user report I was and I traced down the problem. It should now be fixed. If you have been hit by this, please reset your potential duplicates (hit the 'cog' icon on the duplicates page), and then re-search--the bad pairs should not be re-added again.

Several tag autocomplete bugs are fixed today, all for unusual search types. Searching 'hyphenated-namespace:' now gives the correct 'anything' search. 'anything' namespace searches now work for namespaces that have asterisk wildcards. Autocompletes in manage tag dialogs ('write' a/cs) now match results with UPPERCASE inputs. Several more cpu-killer searches (usually some variant of 'get everything' through a clever wildcard) are now impossible to add.

Also, I have fixed the recently borked layout of the 'cancel search' button, which pops up when a page search takes more than three seconds, and fixed a couple of instances where a search page could get stuck in the 'Loading...' state until it was poked.
Open

R: 6 / I: 2 / P: 8

Version 389

https://youtube.com/watch?v=ZO_1d65uRwM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v389/Hydrus.Network.389.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v389/Hydrus.Network.389.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v389/Hydrus.Network.389.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v389/Hydrus.Network.389.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v389.tar.gz

I had a great week. I fixed many small bugs, added some quality of life, and am rolling in updated downloaders for e621 and Deviant Art.

downloaders

Unfortunately, last week's e621 downloader was not getting md5 hashes reliably. This coupled with the URL format change often meant an increased bandwidth load for the subscriptions that were trying to re-find their place. Thankfully, another user has provided a more accurate one that rolls into today's update. It also pulls rating tags. If you paused your e621 subs, please resume them again.

Deviant Art have also been through a layout change recently. The existing file page parser was getting a little shaky, so I wrote a new one this week that uses their API. The default hydrus DA downloader is now wholly API-driven, so it should all be more resistant to any further changes on their frontend. If you would like to search for nsfw DA content, the default DA login still seems to work, but Hydrus Companion is also an excellent option.

For advanced users: Thanks to user contribution, there are two newer default file lookup scripts, an updated iqdb-based danbooru lookup and one for direct danbooru lookup using md5 hash.

For advanced users: The parser String Transformation system now has a 'url percent encoding' option for the encode/decode types. This lets you convert complicated text to a URL-parameter-friendly percent-encoded string, or vice versa, if you need to do something like a clever API URL transformation.
Open

R: 3 / I: 0 / P: 8

Version 387

https://youtube.com/watch?v=YjoL7xy2uA4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v387/Hydrus.Network.387.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v387/Hydrus.Network.387.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v387/Hydrus.Network.387.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v387/Hydrus.Network.387.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v387.tar.gz

I had a great week mostly fixing things and adding and improving small features.

all misc this week

The 'sort files by' dropdown on all pages is now a button. It launches a menu that groups the different sort types, cutting the long list down into easier to navigate groups. Mouse wheel still works on it!

Also, 'sort by framerate' is added. It just does a simple num_frames / duration calculation for now. Fps isn't surfaced in the UI atm, so I expect in the near future to add it to normal file labels and also to add a system search predicate for it.

The options->sort/collect panel finally got its overhaul. Managing namespace sorts is much more sane, support for namespaces with hyphens (like 'creator-id:') is added, and you can edit the order in which they appear in the sort by menu.

Some logic behind tag autocomplete lookup is improved this week. The way special characters like braces and paretheses are handled is better (for instance if you want to search for '[intensifies]'), and now hyphens and underscores are included in these special rules. Typing in 'blue_eyes', 'blue-eyes', 'blue eyes', or 'eyes' will match all of 'blue_eyes'. 'blue-eyes', and 'blue eyes'! Although the results may still be separate without tag siblings, you should never have to worry about searching an underscore version of a tag again. It will take up to five minutes to update your client to reflect the new rules. This is a simple change to a complicated system, so let me know if it fails anywhere, particularly in namespace or wildcard searches!
Open

R: 5 / I: 0 / P: 8

Version 386

https://youtube.com/watch?v=eghGQtQhY38 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v386/Hydrus.Network.386.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v386/Hydrus.Network.386.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v386/Hydrus.Network.386.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v386/Hydrus.Network.386.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v386.tar.gz

I had a great week. I mostly cleaned code, moving old wx definitions and calculations up to Qt format, fixing bugs and colours along the way. There is not much significantly new this week, but I am happy to have cleared out some behind-the-scenes mess.

gif and mpv.conf

Some gifs have metadata that says 'play this once' or 'play this five times' rather than looping infinitely. Hydrus now parses this information, and if you tell it to under options->media, it will obey it.

MPV is extremely configurable. It uses a file called 'mpv.conf' to load its settings. Hydrus has a very bare conf file at the moment, but I expect to expand it as we experiment more with settings that are useful for hydrus. Now you can set the path of the mpv.conf file you want to use, again under options->media. I have added three test mpv.conf files to the hydrus static directory--one for high quality, and two for different kinds of audio normalisation--if you are interested in this subject, or if you have your own mpv.conf files you would like to experiment with, please give them a go and let me know what you think. Once we know what works well for different situations, I will make some more well-rounded profiles for users to choose from.

misc code cleanup notes

You can now edit how fast the mouse cursor autohides on the media viewer, including turning autohide off completely. Yet again, under options->media. The default is 700ms.

The window positioning and sizing system has had several calculation improvements, and it will also now be better at rescuing off-screen windows (for instance if you have a multi-monitor setup that changes between client boots), all with better error handling. Furthermore, some windows like the splash screen--which finally has fixed colours--will now appear on the monitor your mouse is on. If you have had problems with window positioning or sizing, particularly if you have an unusual desktop or multi-monitor setup, let me know if this improves or worsens your situation.

If you are interested in editing your own hydrus QSS stylesheet, please check the install_dir/static/qss directory again this week. I have moved many of the previously hardcoded colours in the client (like green/red dialog button labels) to a new 'default_hydrus.qss' file. This file is loaded by hydrus, and is prepended to any custom stylesheet. You can add these hydrus-specific qss class definitions to your own qss to overwrite them. I expect to continue migrating hardcoded colours to this file in future.
Open

R: 12 / I: 12 / P: 9

Version 385

https://youtube.com/watch?v=lGBLh2iqsZw [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v385/Hydrus.Network.385.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v385/Hydrus.Network.385.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v385/Hydrus.Network.385.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v385/Hydrus.Network.385.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v385.tar.gz

I had an ok week. There are more improvements to mpv and the shortcuts system, and a few misc small jobs done as well.

mpv

Some more general improvements this week:

MPV windows now work in slideshows, and all audios/videos are smoother to move to the next file when slideshows are running.

When you scrub the seekbar below a native or mpv video player, the video should now more reliably pause itself while you drag. Also, more kinds of one-frame music videos (where it is just an image webm with a song in the background) should now appear with their seekbars now, and the seekbars for all these unusual videos and audio files should work better.

A zoom bug where video could navigate down a few pixels on zoom in/out cycles is fixed, and the blank audio mpv window now has some hardcoded zoom capability and should scale itself to fit in the small preview window, always letting you interact with its seek bar.

I attempted to get mpv working for macOS but ran into some technical walls, I think because I am limited to 10.12 Sierra. I do not think there are excellent solutions for this, at least for my macOS expertise, so mpv support on macOS is unfortunately on hold until I learn more.
Open

R: 5 / I: 1 / P: 9

Version 382

https://youtube.com/watch?v=aZL4upd4QpI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v382/Hydrus.Network.382.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v382/Hydrus.Network.382.-.Windows.-.Extract.only.zip
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v382/Hydrus.Network.382.-.Windows.-.Installer.exe
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v382/Hydrus.Network.382.-.macOS.-.App.dmg
source
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v382/Hydrus.Network.382.-.Linux.-.Executable.tar.gz

I had a good but light week. MPV should be working better, and some of the recent UI animation flicker/jitter should be gone.

mpv

Introducing MPV added a weird bug--after the first time an mpv window was opened, the client would then in some cases fail to animate thumbnail fade or process certain intra-program events unless another event, such as a mouse move over the program, restarted the event loop. Some media viewer transitions have also had some single-frame flicker. I have banged my head at this problem for a couple of weeks now, thankfully improving some behind-the-scenes code along the way, but I could not find the full solution. As it happens, rolling Qt back to a slightly older version (5.14.0 to 5.13.0, for users running from source) seems to have completely fixed it. It could simply be a 'native window' bug in the new version of Qt, but at the moment I still suspect my mostly borked wx-code-trying-to-talk-to-Qt-code. Until I can clean that code up, I am happy to have this easier solution.

I fixed a recent issue with the preview viewer not clearing out media correctly on page changes (which was particularly significant now we have audio) and also fixed an mpv crash for Linux/macOS users who happened to already have MPV available through their OS. Also, MPV is now configured to have a 'lighter touch' on filesystems, so if you have a client with a laggy file storage system (e.g. a very large client stored over a network share) and have experienced extreme latency when trying to open MPV, this situation should be much better. MPV should be fairly stable now, so my big remaining tasks are to get nicer volume controls and proper build support for Linux and macOS.

MPV as a program relies on '.conf' files for most of its configuration. Hydrus now follows this format. If you have your own conf or otherwise would like to edit this, please check out the mpv.conf under install_dir/static/mpv-conf. If you have settings you think should be in the default for all users, let me know. I'll likely let you optionally override where hydrus looks for its conf (where it can't be accidentally overwritten on update) in future.
Open

R: 11 / I: 3 / P: 9

Version 381

https://youtube.com/watch?v=sz8xLRCHP_k [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v381/Hydrus.Network.381.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v381/Hydrus.Network.381.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v381/Hydrus.Network.381.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v381/Hydrus.Network.381.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v381.tar.gz

I had a good week with a couple of challenges. MPV is now ready for all windows users and is turned on by default.

MPV

Thank you to the advanced users who tested and gave feedback on MPV. I have eliminated the crashes, tightened up the jank, and am now rolling it out to all Windows users by default for video, audio, and gif/apng. All media view settings under options->media will be reset this week.

MPV is a good free media viewer. One of the core benefits of moving hydrus to Qt was being able to plug it into our media player, enabling hardware-accelerated video playback and audio. It looks just like the native player, with the seek bar beneath, but it works much faster, able to play 1080p or 4k videos at 60fps at full or unusual zooms. And of course, it makes noise!

This is early days. I have only just started plugging into MPV, so many features are basic or not yet available. The global volume and mute controls are currently some very ugly controls in the top media hover panel. Slideshows will not move forward on an MPV window (hydrus doesn't know when an MPV player has 'played once through' yet), and some processes like the archive/delete filter will need some extra workflow options now that more users will be playing videos at high res (left-click on the player pauses the video, so to set 'keep' on archive/delete, you'll have to click on some whitespace, of which there is so much less when the video is so big). Please let me know what your top priorities for improvement are, and of course, if you encounter errors or crashes, let me know. I'll keep working.

Even though it was a lot of difficult work, I am overall really pleased with how this has gone. The only big remaining bug that I need to nail down is an unusual thing where after multiple mpv viewings in a preview window, that page will stutter some query/thumbnail loading unless the mouse is moving. This is a slightly frustrating bug, but the benefits of MPV are enough that I am happy to live with it. I will also get it going for Linux and macOS, which I did not have time for this week.
Open

R: 15 / I: 1 / P: 9

Version 380

https://youtube.com/watch?v=U35sSPJI_Bs [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v380/Hydrus.Network.380.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v380/Hydrus.Network.380.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v380/Hydrus.Network.380.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v380/Hydrus.Network.380.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v380.tar.gz

I had a couple of difficult weeks, with illness and other IRL problems getting in the way, but I got some hopefully pretty neat work done. A new fast video and audio player is available for advanced users to test, and there are a bunch of fixes and ui improvements as well.

A user just notified me that the Duplicates page has crazy layout! I apologise. Nothing is broken, it is just sizing wrong, and part of a longer fight I am having to convert my old wx layout code to Qt. I know exactly what happened here, and I will have it fixed for 381. If you discover more UI like this (the system predicate panels have a bit of it), please let me know.

R: 1 / I: 1 / P: 9

Invitation to Endchan's Fourth Christmas Special!

Discussion thread:
>>>/operate/10818

Thanks.
Open

R: 28 / I: 4 / P: 9

Version 379

https://youtube.com/watch?v=nvy6cHhpZdQ [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v379/Hydrus.Network.379.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v379/Hydrus.Network.379.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v379/Hydrus.Network.379.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v379/Hydrus.Network.379.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v379.tar.gz

Happy New Year! Although I have been ill, I had a great week, mostly working on a variety of small jobs. Search is faster, there's some new UI, and m4a files are now supported.

search

As hoped, I have completed and extended the search optimisations from v378. Searches for tags, namespaces, wildcards, or known urls, particularly if they are mixed with other search predicates, should now be faster and less prone to spikes in complicated situations. These speed improvements are most significant on large clients with hundreds of thousands or millions of files.

Also, like how system:inbox and system:archive 'cancel' each other out, a few more kinds of search predicate will remove mutually exclusive or redundant predicates already in the search list. system:limit predicates will remove other system:limits, system:audio/no audio will nullify each other, and--I may change this--any search predicate will replace system:everything. I have a better system for how this replacement works, and in the coming weeks I expect to extend it to do proper range-checking, so a system:filesize<256KB will remove a system:filesize<1MB or system:filesize<16KB or system:filesize>512KB, but not a system:filesize>128KB.
Open

R: 11 / I: 3 / P: 9

Version 378

https://youtube.com/watch?v=gTFG_nvreoI [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v378/Hydrus.Network.378.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v378/Hydrus.Network.378.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v378/Hydrus.Network.378.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v378/Hydrus.Network.378.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v378.tar.gz

I had a great, simple week. Searches are less likely to be very slow, and system:limit searches now sort.

all misc this week

I identified a database access routine that was sometimes not taking an optimal route. Normally it was fine, but with certain sizes or types of query, it could take a very long time to complete. This mostly affected multi-predicate searches that included certain tags or system:duration and system:known urls, but the routine was used in about 60 different places across the program, including tag and duplicate files processing. I have rewritten this access routine to work in a more 'flat' way that will ensure it is not so 'spiky'.

Also in searching, I managed to push all the 'simple' file sorts down to file searches that have 'system:limit'. If you search with system:limit=256 and are sorting by 'largest files first', you will now see the 256 largest files in the search! Previously, it would give a random sample. All the simple sorts are supported: import time, filesize, duration, width, height, resolution ratio, media views, media viewtime, num pixels, approx bitrate, and modified time. If you want something fun, do a search for just 'system:limit=64' (and maybe system:filetype) and try some different sorts with F5--you can now see the oldest, smallest, longest, widest, whateverest files in your collection much easier.

There are also some fixes: if you had sessions not appearing in the 'pages' menu, they should be back; if you have had trouble with ipfs directory downloads, I think I have the file-selection UI working again; 'remove files when trashed' should work more reliably in downloader pages; and several tag and selection lists should size themselves a bit better.
Open

R: 9 / I: 2 / P: 9

Version 377

https://youtube.com/watch?v=zXaFARzFG3c [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v377/Hydrus.Network.377.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v377/Hydrus.Network.377.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v377/Hydrus.Network.377.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v377/Hydrus.Network.377.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v377.tar.gz

I had a good week. I mostly caught up with my smaller jobs queue, just pushing on work that had piled up.

Qt

Thank you for the continuing Qt bug reports. I fixed a variety of typo-broken buttons this week, mostly buried in less-used UI, and if you have set a browser launch path override in the options, hyperlinks across the program should obey that again. I also believe I fixed the annoying issue where media viewer hover windows that needed to shrink (because of switching to a new media that had fewer info lines or known urls) would linger too tall for one frame. EDIT: I'm still having slight hover window resize flicker in my IRL client when I keep my mouse over the top-right hover, I'll give it another go next week.

Furthermore, all non-menubar menus across the program now open on right-click release instead of press. This sounds trivial, but there was a mix of down/up click handling here, now unified, and some things like thumbnail and tag selection are handled more naturally in the initial click down. This also fixes an issue where users who had menu styles with no significant border were sometimes accidentally activating the first menu item on a slow click (the click down would show the menu, the click up would activate it).

Last week's support for styles and stylesheets for Qt seems to have gone well. There is more to do here, but this week brings better error handling for unusual situations, better 'we already have this set, no need to waste CPU re-applying it' detection, and fewer hacky colour overrides, so custom stylesheets should be able to colour regular panels without lots of unintended grey or white boxes.

the rest

The shortcut system should now be able to support any text key. This includes extended characters like รธ or รฆ. If you need to apply a modifier to get these keys (e.g. ctrl+alt+e), these modifiers are likely to show up spuriously in the shortcut's label (e.g. ctrl+alt+รฆ), but everything should nonetheless line up in the program correctly. Let me know if you discover any new bugs here!

There are also some new shortcut commands this week, for the 'media viewer' set. They are 'pan_top/bottom/left/right_edge' and 'pan_horizonal/vertical_center'. They move the whole image around to edges, or center it again, and should be helpful if you often look at tall or wide comics at 100% zoom. There will be more to come here in coming weeks.

The work on making the UI smoother is also continuing. Session saving and a variety of 'media refresh' events that occur on file import or maintenance are now significantly faster and do the bulk of that work in the background. Users with very large sessions should experience less juddery behaviour during both idle and heavy work periods.

R: 0 / I: 0 / P: 9

Release Tomorrow!

I had a good week. I focused on catching up on smaller work. More bugs are fixed, there's some neat new shortcuts to pan media around, the UI should be smoother when importing files and saving sessions, and the system:known urls predicates now run much faster.

The release should be as tomorrow.
Open

R: 11 / I: 5 / P: 9

Version 376

https://youtube.com/watch?v=J4wKjGHvs_4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v376/Hydrus.Network.376.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v376/Hydrus.Network.376.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v376/Hydrus.Network.376.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v376/Hydrus.Network.376.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v376.tar.gz

I had an ok to good week. Subscriptions run better and there are more UI fixes and basic theming for Qt.

subscriptions

Subscriptions have always been resource heavy. As last year's downloader overhaul extended their capabilities, we saw increased CPU lag with many subscription operations. I had hoped to completely fix this this week, even permitting subs with >250,000 items cached, but the surgery needed to figure this out was more complicated than I expected. While I otherwise prepped for this next step, I also tightened up subscription timing code and wrote a new scheduler that reduces overall subscription resource needs.

Subscriptions will now try to load when they are due, and only then. So, if a query in your sub is due to check in 45 minutes, or if it has file to download but has run out bandwidth for 45 minutes, it should now run, typically, in 45-60 minutes. They will also respond better to cancelling and pause events, and if only 30 of 100 queries are due for a check in this run, or have outstanding files to download, you will now see x/30 on the popup progress text and gauge, just as better indication of how much work remains.

Previously, subs checked every four hours. It meant a bunch of wasted CPU for subs that were not due, late checks for subs that were due just moments after the check time, and also a delayed 'clumping up' of work. Today's change is subtle but significant, so I am interested in feedback from all users of subs--if you notice anything odd, bad, or great, please let me know. Ideally, subs will be quieter for you, running more often when you are not looking.

Although the data overhaul was too much work to do in one week, I now have a good plan and am in better position to get it done. I really want subs to load and save their metadata in milliseconds so the manage subs dialog can launch equally quickly and the client api can start talking to them. The year is fading, so I am not sure when this will get done, but hopefully this week's scheduling will at least relieve some of the work involved.

R: 3 / I: 0 / P: 10

QSS Stylesheet Testing

Was asked to forward this here.
Hydrus using PySide2, QSS & CSS Stylesheets currently 'mostly' work as intended. However, all Sub-Panels including Docked Panels (media browser and the like with a centered title), Hover Panels (media viewer panels for example) ignore styling in QSS or CSS formats.

Here's a handful of linked Stylesheets to try during testing: https://wiki.qt.io/Gallery_of_Qt_CSS_Based_Styles

Qt Dark Orange was used in the image.

R: 0 / I: 0 / P: 10

Release Tomorrow!

I had an ok week. Subscriptions now operate more efficiently, using less resources and choosing their sync times more precisely. Some simple Qt theming support is added for experimentation by advanced users, and a number of bugs are fixed, including page tab drag and drop for macOS users and anyone else with a center-aligned tab bar.

I have a busy day, so the release will be quite late tomorrow.
Open

R: 7 / I: 0 / P: 10

Version 375

https://youtube.com/watch?v=M5DqEHxY4FM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v375/Hydrus.Network.375.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v375/Hydrus.Network.375.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v375/Hydrus.Network.375.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v375/Hydrus.Network.375.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v375.tar.gz

I had a great week. There are a bunch more Qt fixes, and a few other things as well.

Qt

I have fixed a bunch more bugs in the Qt code. We are getting to the end now--this is mostly smaller stuff like an unusual dialog button not working, but I have fixed another important memory leak that was causing some backend not to be deleted correctly when a media viewer closed on a video. This should radically reduce memory use for some heavily used clients.

Some windows that were large, or could expand to be, like the options dialog on some pages, were sizing off the edge of the screen. This should be fixed now, and a variety of child-window initial size calculations should be a bit more accurate--Qt manages window size a little differently than wx, and the additional buffer represented by the window title frame and border was not being taken into account.

The menubar menus should work a little snappier this week. Things like the rapid 'pending (xxx)' menu updates when a downloader is importing files with tags. The whole UI should get a little latency benefit from this during these high-traffic times.

I also gave the layout and scaling another go on clients that use UI scaling (when you tell your OS to display UI at >100% on a monitor) on high dpi displays. The experiment from last week did not go how I wanted, with pixelly scaled-up thumbnails, so I have reworked that and fixed the taglist, which was crunching tags together on these displays.
Open

R: 0 / I: 0 / P: 10

Cant migrate?

trying to move my database and I get this error when I click "move entire database and all paths":


AttributeError
'DirDialog' object has no attribute 'SetPath'
File "include\ClientGUICommon.py", line 199, in EventButton
self._func( *self._args, **self._kwargs )
File "include\ClientGUIScrolledPanelsReview.py", line 551, in _MigrateDatabase
dlg.SetPath( source )
Open

R: 18 / I: 4 / P: 10

Version 374

https://youtube.com/watch?v=t5jroSCBBwk [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v374/Hydrus.Network.374.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v374/Hydrus.Network.374.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v374/Hydrus.Network.374.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v374/Hydrus.Network.374.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v374.tar.gz

I had a great week. A ton of Qt problems are fixed, and a macOS App is ready. If you were waiting for a cleaner release, I would recommend this for all Windows and Linux users.

Qt

I mostly worked this week on Qt bugs. I appreciate all the reports everyone sent in. I have fixed a whole lot, mostly bringing things back in line to where the wx build was. The whole list is in the changelog, but the highlights are:

Fixed the cursor not unhiding in the media viewer as long as it was over a media.

Fixed the resizing texts that were causing subscription popups and others to bounce around.

Fixed pages auto-resorting media on creation when not desired.

Fixed tab drag and drop when 'do not follow' (+shift) mode is on.

File drag and drop to discord should be fixed on Windows if the options->gui BUGFIX is set.

Added basic high DPI scaling support--all feedback appreciated.

Fixed some memory leaks.

There is still some stuff I didn't have time to get to--some layout/sizing problems for dialogs, some splitter/sash positioning issues, focus sometimes not transferring when requested, and some weirder stuff like html rendering as a web document in text labels rather than displaying literally. I'll keep going, but the vast bulk of the work is done.

R: 0 / I: 0 / P: 10

Release Tomorrow!

I had a great week. I have fixed some more Qt problems, including some bad window positioning, thin popup message widths, a video memory leak, and another round of improvements for high-dpi displays. Also are some improvements to the file maintenance system for recovering (including auto-redownloading) missing files, some better network retry timing code, and new downloaders for some broken sites.

The release should be as normal tomorrow. /hydrus/ is back up on 8kun, and I will be making a release post there as well as here.
Open

R: 56 / I: 13 / P: 10

Version 373 (Qt)

https://youtube.com/watch?v=2IEyVp6lMpY [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v373/Hydrus.Network.373.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v373/Hydrus.Network.373.-.Windows.-.Installer.exe
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v373/Hydrus.Network.373.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v373.tar.gz

The Qt update is ready for Windows and Linux!

This week's release is for all users, but please bear in mind it has some small layout and positioning bugs, such as subscription popups sizing a little strangely. If you would rather wait a week or two for these last issues to be cleared (and any others that pop up as more people play with this), that is totally fine.

Qt background

Since hydrus began as an application, it has used wxWidgets to draw all the windows and buttons on screen. wx has served us well, but hydrus has grown to be a complicated program with hundreds of different custom things going on, and it was starting to show. Lots of windows were flickery, modern tech like 4k screens were not excellently supported, and operating systems and window managers were unstable. If I could have moved to a more flexible and more frequently updated UI library by snapping my fingers, I would have, but the total UI code is almost three megabytes, far too much to reasonably convert as I kept at my normal weekly schedule.

A user contacted me I think about a year ago talking about Qt and possibly making some scripts to automatically convert hydrus's wx code to Qt. I said it sounded like a good idea, and he worked in the background trying to figure it out and add manual tweaks. He was very successful, ultimately getting an essentially functional build going a couple of months ago. He passed the code to me four weeks ago, and I have since crash-learned Qt and fixed the great majority of the bugs that slipped through the automatic conversion process.

I am extremely grateful for this user's work--this would not have happened otherwise--and I am very happy with the result. Qt is a nicer library than wx for our purposes, runs faster, has much less flicker and related jank, and provides many new options for future extensions and customisation. I also enjoy working with Qt--the library is good.

R: 0 / I: 0 / P: 10

Release Tomorrow!

I had a great week mostly working on a variety of bugs in the new Qt build. The majority of the jank and annoyances are fixed, performance and memory use are improved, and I have a macOS release ready.

The release should be as normal tomorrow.
Open

R: 2 / I: 2 / P: 10

Test Build of 373

THIS IS FOR ADVANCED USERS ONLY. IF YOU ARE JUST A REGULAR USER LOOKING TO UPDATE, THE PROPER RELEASE WILL BE OUT ON THE 13th.

Hey, here are test builds for Windows and Linux:

https://mega.nz/#F!ojwgFa6C!ycjHGcP7FLDYe6hXNO16GQ

DO NOT USE THESE ON A REAL CLIENT. THERE ARE BUGS, AND YOU CANNOT ROLL BACK TO 372!

If you are interested, please feel free to extract one and try out a fresh client for five minutes. I have already received plenty of feedback on smaller layout issues and some broken controls, so I wonโ€™t waste your time asking for smaller reports, but if you are unable to run the client or import a handful of typical files, I would be interested to know what you get and what OS you are on.

Please forgive me for being late to replying to messages for the next week. There is still a good bit to do, but I feel good about having a decent release ready for the 13th.

R: 3 / I: 0 / P: 10

wx->Qt Status Update 1

I had a good week. I continued to work on the Qt version of hydrus, fixing bugs and cleaning code. There is still a good bit to do. I now have working builds for Windows and Linux. I have a working OS X environment but need to put more work in to get a running App.

I still feel good about the schedule. I hope to have 373 ready for the 13th November. I will put out a preview build for advanced users to give feedback for on the 6th.

R: 0 / I: 0 / P: 11

Qt Release Tomorrow!

I had a great and very busy four weeks. I appreciate everyone's patience.

The hydrus client is now switched over to a more stable and feature-rich Ui library, Qt. This is a big and important change behind the scenes, but the client looks and works essentially the same as before, though smoother and less flickery. The update process should be as normal--no special instructions. I also folded in a little normal work, including an important fix to the file maintenance system and some quality of life gui improvements.

The release is likely to be late tomorrow, as I want to do some more testing on my own system and with advanced users. I have a working macOS build, but it has some sizing/layout problems still to fix, so if I cannot nail them down, the macOS release will be delayed a week for 374.

R: 1 / I: 0 / P: 11

wx->Qt Status Update 3

Things are going well with the update. I am coming to the end of my โ€˜must do before a releaseโ€™ todo list. I feel good about having a decent release ready for the 13th, and hopefully with a couple of new bells and whistles thrown in.

I would still like to do a test release for advanced users, mostly to test that my builds will at least boot on various systems and there are no very unpleasant surprises. I had thought to do this later today, the 6th, but there are still some important things I want to do, and I do not want to rush it out just for the sake of it. I will now aim for Saturday the 9th.
Open

R: 18 / I: 12 / P: 11

Version 372

https://youtube.com/watch?v=Klw-5eCji1w [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v372/Hydrus.Network.372.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v372/Hydrus.Network.372.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v372/Hydrus.Network.372.-.OS.X.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v372/Hydrus.Network.372.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v372.tar.gz

I had an ok week. I mostly fixed bugs and added little things to make for a 'clean' final wx release.

This is the last release for approximately four weeks.

all misc this week

You can now customise the height of the media viewer's animation scanbar, and the width of its nub, under options->media.

Right-clicking on a single tag in a taglist now lets you add/remove it from your current favourites. The whole menu there got a bit of cleanup.

I added a link to https://gitgud.io/koto/hydrus-archive-delete to the Client API help. It is a web browser based archive/delete filter that plugs into the Client API.

The new tag filter favourites system now has buttons to import/export tag filters from/to your clipboard. You can now share complicated filters with other clients or friends.

A couple of file import 'publishing' operations--when thumbnails from an import queue or subscription get added to a page--are now a little less laggy.

If global network traffic is paused, subscriptions will now not start work.

R: 4 / I: 0 / P: 11

wx->Qt Status Update 1

I had a good week. I received the Qt code from the user who put the conversion work in and have familiarised myself with the very basic differences between wx and Qt. The client boots and runs well. I also managed to create a working build for Windows today, so there do not seem to be any huge technical hurdles to getting this done.

There are still things for me to learn, bugs to deal with, and miscellaneous work (fixing layout issues and so on) until the release will be ready. I feel very good about things and believe I am on schedule to put out 373 on the 13th of November. I will put out a preview build for advanced users to give feedback for on the 6th.
Open

R: 11 / I: 1 / P: 11

Version 371

https://youtube.com/watch?v=aZClzeSIVNA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v371/Hydrus.Network.371.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v371/Hydrus.Network.371.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v371/Hydrus.Network.371.-.OS.X.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v371/Hydrus.Network.371.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v371.tar.gz

I had a good week. There are several fixes, and I have added 'favourites' management for tag filters.

favourite tag filters

The Tag Filter is the newish whitelist/blacklist panel that manages which tags will be parsed, blacklisted, migrated, or, more recently, hidden on the new tag display system. It is being used in more and more places, so I have added the ability to save and load certain filters for quicker re-access.

So, now every tag filter edit panel has save/load/delete buttons in the top-right. They all share the same list of favourites. I also put in some example defaults. Let me know if you run into any trouble with it!

the rest

Thumbnails now sort by 'number of tags' and namespace sorts using the 'single' tag display context. So, if you have your thumbs set to hide page tags from the PTR, these page tags will no longer be included in a 'creator-series-volume-chapter-page' sort, for instance.

I fixed several smaller issues--a kind of video that would not import correctly, an unusual 'options' saving error after setting a number of ratings on several thumbnails, seeing duplicate thumbnails on some downloader highlights, gallery downloads that stopped early due to malformed post URLs, download pages sometimes secretly performing a collect and sort when a default collection is set, the media viewer being unable to show certain malformed files, and remote thumbnails not refreshing correctly when their respective file is imported.

Advanced users only: URL Classes now let you set whether requests for URLs matching the class will send a 'referer' http header. Normally, hydrus will send any 'previous' URL as the 'referer' (typically the gallery page when visiting a post page, or the post page when downloading a file), just as if you were visiting it in your web browser, but you can set it to never send any referrer, always use an auto-generated referrer, or use the auto-generated one when no natural referrer is available (which can happen on a user drag-and-drop URL import). The auto-generated referrer is a standard StringConverter operation on the URL being visited itself--if you can figure out a way to infer it from the original URL, or just replace it with a static value, I think some more perculiar downloaders should be possible.

R: 1 / I: 0 / P: 11

I had an ok week. I mostly fixed bugs to make a nice 'clean' final release in wx.

The release should be as normal tomorrow. It will be the last release for about four weeks, as the next job will be finalising an large and long-planned wx->Qt (UI library) conversion.
Open

R: 22 / I: 2 / P: 11

Version 370

https://youtube.com/watch?v=h27wMuv_--s [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v370/Hydrus.Network.370.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v370/Hydrus.Network.370.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v370/Hydrus.Network.370.-.OS.X.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v370/Hydrus.Network.370.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v370.tar.gz

I had a great week. There's a big improvement to tag display management and a bunch of smaller stuff.

tag display

Part of this ongoing tag overhaul work I am doing is to catch up some ancient systems that aren't able to deal with the size of the dbs we have now. The old 'tag censorship' is probably the biggest example of this. I have rewritten this to work in a more sensible way using newer objects.

The main difference is services->manage tag censorship is now service->manage tag display. You can now hide tags from the front UI using the new Tag Filter interface, with separate filters for 'single' views, like thumbnail banners and the taglist on the media viewer, and 'multiple' views, which for now means the 'selection tags' list on normal search pages. For instance, you might like to hide page: tags from the selection tags list, since they usually clutter things, or you might like to only show creator/series/character. You can set any normal whitelist or blacklist, and filtered for each tag service (e.g. 'only show creator/series/character tags from the PTR') and/or blanket rules for all services.

Right-clicking on a tag in one of the lists will provide options to hide it or its namespace from that list type. These menu actions now take effect immediately!

R: 0 / I: 0 / P: 11

I had a good week. I mostly fixed things and cleaned code, and I added 'favourite' tag filters to make it easier to manage and reload common filters.

The release should be as normal tomorrow.
Open

R: 25 / I: 5 / P: 11

Version 369

https://youtube.com/watch?v=ACAvBBMb6BM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v369/Hydrus.Network.369.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v369/Hydrus.Network.369.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v369/Hydrus.Network.369.-.OS.X.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v369/Hydrus.Network.369.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v369.tar.gz

I had a good week. The file maintenance system works better, and files now know their 'modified' dates.

file maintenance

The new file maintenance system now works smarter and more productively. Previously, it did rare bursts of hard work in idle time, and now it does small amounts of work all the time. You do not have to do anything--it will now keep up with its behind-the-scenes queues more quickly and quietly, hopefully without giving you any hassle. The only significant difference is file maintenance no longer occurs on shutdown. If you are not interested in any details, you can skip the following completely.

Its settings under options->maintenance and processing have changed a little--rather than the old '200 files per day' default, it now has faster '1 work unit per 2 seconds/20 seconds' for idle/normal time. These values should not tax your system or interfere with other programs or interrupt your browsing. If you have a faster or slower computer, feel free to change them. If the work does make your browsing juddery, please do turn off work in normal time, which you can do in the options or now more quickly at database->maintain->file maintenance. The 'work unit' is part of a new weighting system that lets fast jobs--like checking for file presence on the hard drive--count for less work in the throttle and process faster than bigger jobs like regenerating thumbnails

When I added the 'go through all video files and scan for has_audio property' job some weeks ago, it added 100,000+ jobs for many users. The 200/day default was not cutting it, so this new 'lots of light work all the time' mode should get these big jobs done in reasonable time without going nuts. It also updates thumbnails in real-time, so there is the smallest chance you'll see one of the new 'has audio' icons fade into one of your thumbnails right as you look at it! The new processing also helps for another large retroactive job being added this week--figuring out file modified dates:

R: 0 / I: 0 / P: 11

Release Tomorrow!

I had a great week. As well as a variety of fixes and little improvements like new 'duration' icons on thumbnail and system:modified date, I finished the first version of the new tag display/filtering system. You can now hide specific tags and namespaces from the 'tag selection' and media viewer lists separately, and a variety of tag tasks like media selection in a big thumbnail view are a bit faster. There is also an important fix to the new asynchronous tag repository processing.

The release should be as normal tomorrow.
Open

R: 1 / I: 0 / P: 12

Extension Thread

ITT; extensions and thing you can bolt on to Hydrus to make it better

http://archive.fo/rmx9d
http://archive.fo/C8ylc

R: 1 / I: 0 / P: 12

Co-development generals thread

Due to the recent take down of ExHentai, is pretty alarming that the same might happen to similar sites. So I propose to anons using Hydrus to save the doujins they like within the platform and tag them along so whenever Hydrus reaches an stage where sharing is easy, maybe you'll have not only a decent alternative to Sad Panda, but a lasting one.

Numerous ideas have been thrown around about it, but here is what the BO have confirmed:
1) Hydrus is not currently great at handling paged media. It can do it, but the workflow is a little awkward. I plan to improve this in future.
2) IPFS support is prototype, and mostly for advanced users who are already comfortable with the program. I'm working on it now.
3) Hydrus is not for everyone. If you have 10,000+ files and want to manage them better than files and folders, you may like it. If you want a clean and professional experience with beautiful UI, you may not.

So it's nothing but a project now. The main reason I do the thread is to keep anons up to support it and use Hydrus for managing whatever manga they downloaded to be used later when the project get to the point it can be used and to post resources and the such.

An index with tags and names:
https://mega.nz/#!dBxmUADB!4piJItp7ja7_9WbTmoLGo3nMLOP2Fr0AH_ju9W4-PLY

Hydrus+IPFS that supports concurrency
https://github.com/sakharovaan/hydrus-unleashed
https://github.com/rr-/szurubooru

IPFS+TOR/I2P is in the works
https://github.com/OpenBazaar/go-onion-transport
https://github.com/cretz/tor-dht-poc
https://github.com/RTradeLtd/Temporal
https://github.com/RTradeLtd/go-ipfs-plugin-i2p-gateway
https://github.com/RTradeLtd/ipfs-i2p-bridge
https://github.com/RTradeLtd/go-ipfs-plugin-i2p-bootstrap

Old thread: http://archive.fo/C2xLp
Open

R: 13 / I: 2 / P: 12

Version 368

https://youtube.com/watch?v=KhWArBhPWu0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v368/Hydrus.Network.368.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v368/Hydrus.Network.368.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v368/Hydrus.Network.368.-.OS.X.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v368/Hydrus.Network.368.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v368.tar.gz

I had a great week. The PTR has moved successfully, and I got multiple local tag services working.

PTR has moved

The Public Tag Repository has changed management. I am no longer running it or involved in working as a janitor. There is more information here:

>>62

As a result, the PTR no longer has bandwidth limits! The user now running it is also putting together a janitorial team to catch up on petitions as well. About ten million delete mapping petitions and six thousand add sibling petitions had piled up! If you would like to talk to the new management, they are available on the discord. The current plan is to keep running the PTR with the same loose rules as I did--the main concern was overcoming my bandwidth limits.

If you currently sync with the PTR, you will be given a yes/no dialog when you update asking if you would like to keep using the PTR at the new location. If you select yes, the client will automatically update your service (the only credentials difference is that instead of it being at hydrus.no-ip.org, it is now at ptr.hydrus.network), and you will keep syncing and be able to continue uploading without skipping a beat. If you select no, your PTR stay as-is but will pause. If you still sync with my read-only test file repository, this will be paused automatically.

While I am very thankful for the 650 million submissions I have had to the PTR over the past seven years, it is a load off my mind to no longer be responsible for it. I much prefer being a developer than an administrator and hope to use the extra time and upcoming feedback to work on improving the admin side of hydrus repositories, which, due to me being the primary user, have always had debug-tier UI and half-broken features.

The various areas in the help have been updated to reflect that the PTR is no longer mine, and the quick setup under the help menu is now just help->add the public tag repository.

R: 0 / I: 0 / P: 12

Release Tomorrow!

I had a good week. As well as some smaller fixes and improvements, I added file 'modified date' metadata to the client, letting you see and sort by it, and improved the file maintenance system (which will retroactively generate modified date info and catch up on the existing 'has audio' detection queue) to work quietly all the time in the background, so its gets much more work done with much less hassle.

The release should be as normal tomorrow.
Open

R: 13 / I: 0 / P: 12

Version 367

https://youtube.com/watch?v=Z-eZl_CVQpA [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v367/Hydrus.Network.367.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v367/Hydrus.Network.367.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v367/Hydrus.Network.367.-.OS.X.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v367/Hydrus.Network.367.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v367.tar.gz

I had a good week getting more of the tag migration working and adding some audio and ratings ui features.

sibling and parent migration

Last week's tag migration dialog now supports sibling and parent migration! Just like with tag mappings, you can copy between any combination of tag services and an external file type called a Hydrus Tag Pair Archive. There are tag filters for the left and right side of the pairs being migrated, so you can easily capture all the character->series parents or unnamespaced->creator siblings, for instance.

I am happy we finally have a way to import and export siblings and parents en masse. This should make it much easier to move tag pairs between local tag services and tag repositories, and, say, for an enterprising user to pull all the tag aliases and implications from a booru and get them into hydrus.

R: 6 / I: 1 / P: 12

The PTR will undergo a change of management in two weeks

I have run the PTR since the start of hydrus, seven or so years now. I remember hitting the first 1,000 tag mappings, then being stunned at 1,000,000, and now we are at 650,000,000. It was always meant to be a test prototype, but I kept it going because it was so neat seeing the tech work, getting positive feedback, and dealing with all the interesting problems millions and then hundreds of millions of tags presented.

Unfortunately, as it has grown, it has become increasingly unwieldy to manage by myself. I have fallen behind on dealing with petitions, and I have increasing bandwidth problems you have no doubt noticed. I have never been an excellent server administrator, and I don't like being a single source of failure, either. A user who has more bandwidth has kindly offered to run it in my stead, so therefore I will be shutting it down on my end, likely on September 16th, in time for version 368.

I will sanitize the database so it has no more information than a syncing client currently knows (there have never been IP logs or anything, but I will do some additional work such as deleting the petition 'reasons' users have submitted over the years, some of which have personal joking messages to me) and put the frozen database up on a Mega account with scripts to generate new admin access keys and so on. Any user who wants to download it and continue their own tag repository with its data, whether that is private or public, will be able to. I will also create some Hydrus Tag Archives for the tags and parents and siblings for users who are separately interested in that data.

R: 0 / I: 0 / P: 12

Release Tomorrow!

I had a great week. The PTR has moved to its new higher bandwidth location and management successfully. If you sync with the PTR, tomorrow's client will offer to automatically move your settings to the new location and you will be able to continue syncing and uploading tags without skipping a beat. Also, I was able to get multiple local tag services working completely!

The release should be as normal tomorrow.
Open

R: 1 / I: 0 / P: 12

>>86
nope, running windows 7 and from installer, no idea why it wont import for you, but it was already in my db and not really sure where I would find it again to dick around.

also

I got somewhat done with the image thing I was doing

https://mega.nz/#!Dl4ABYwL!t53V4rjD7nqPYrPq-PYDT4ug-Uo67B0OfvnCOIpXdqc

its I think 9 images, with source files where I ran them through waifux2 at 1.0 magnification trying to see where they started to fall apart in quality, some of them go fairly low before really shitting the bed, some of them aren't able to handle the conversion to jpeg at all.

one of the images I went x2 on source because I have seen a lot of jpegs where they are twice the resolution, about the same file size but far higher quality then the one that should have been higher quality.

the main reason I did this was a thought. now, i'm not sure about you but I have a fuckload of images that I could run though this and get a reasonable file out of, but the only way to do this in bulk is to copy every image from the archive out, convert them 1 by 1 and zoom in on them which is honestly a shit work flow.

if this was capable in hydrus, you could have the source image convert the image, then save both images to a duplicate detector, hell if you wanted to get anal about it, you could have them go
20 - 40 - 60 - 80 - 100 and you pick where the image falls apart, then from there it goes into more fine grain. like lets say 40 was where it fell apart, then it would explode 40-60 with 40-45-50-55-60 and you figure out where the image goes to shit. most images seem to be good with about 1/5 to 1/10 their file size.

This would also put waifux2 in the program, so images that are fucked that you like could potentially be fixed inside.

each one of these going through a 'dup detector' assigned to conversions, so you can manually tell it 'yea, this worked' as this workflow is far less of a dick pain as trying to compare source to conversion out of program, you could also, in the case of the overwatch image, likely shrink it in waifux2, haven't tried, shit took up way to much ram to work with that monstrosity.
Open

R: 18 / I: 2 / P: 12

Version 366

R: 0 / I: 0 / P: 12

Release Tomorrow!

I had a good week. The new tag migration system now supports siblings and parents, including import/export to external files, and I added shortcuts to increment/decrement numerical ratings, and added sort and thumbnail display for the new 'has audio' property.

The release should be as normal tomorrow.
Open

R: 16 / I: 1 / P: 13

Version 364

https://youtube.com/watch?v=oqRlJC-ckL4 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v364/Hydrus.Network.364.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v364/Hydrus.Network.364.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v364/Hydrus.Network.364.-.OS.X.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v364/Hydrus.Network.364.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v364.tar.gz

I had a difficult week, but I got some nice work done. There is a new way to add complicated tag seaches, and repository processing is now a lot easier to work with.

complicated tag searching

Users in help->advanced mode will now see a new 'OR' button on their search pages' autocomplete dropdowns. Click this, and you will be able to paste a logically complicated pure-text query such as "( blue eyes and blonde hair ) or ( green eyes and red hair )", and it will automatically convert it to a tag query hydrus can understand! (It converts it to conjunctive normal form, or ANDs of ORs)

This code is thanks to user prkc, who wrote a really neat parser to do the conversion. The system even supports XOR! Complicated searches will produce similarly crazy hydrus search predicates--which may in some cases run a little slow--so give it a go and see how you like it. This is mostly a first prototype to see how it goes in the real world, and I am happy to revisit it based on feedback.
Open

R: 1 / I: 0 / P: 13

>> new board, how do
>> find suggestions etc
>> cry a lot
>> duration for vids a thing maybe?
>> nope?
>> could it be ... push toward vid/audio support maybe, sometime when?
>> plead with the hydrus god's

R: 0 / I: 0 / P: 13

Release Tomorrow!

I had a great week finishing off what I started last week. There is now a comprehensive way to move and delete large numbers of tags between services and external archive files. I also fixed a variety of bugs, reworked the main right-click menu on media (including extensions for the 'remove' command, such as 'remove all archived'), and added a couple of shortcuts to perform mass-page refreshes.

The release should be as normal tomorrow.
Open

R: 0 / I: 0 / P: 13

Hello dev
Is there filesize limit in hydrus?
Hydrus yields an error when I'm going to import a file bigger than 5GB
Of cource I have enough space on HDD

Here's error message and I'm using Hydrus ver 365 and Arch linux
filesystem is ext4

OSError
[Errno 28] No space left on device
Traceback (most recent call last):
File "/home/erdos/ssd/hydrus/include/HydrusPaths.py", line 719, in MirrorFile
shutil.copy2( source, dest )
File "/usr/lib/python3.7/shutil.py", line 266, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.7/shutil.py", line 122, in copyfile
copyfileobj(fsrc, fdst)
File "/usr/lib/python3.7/shutil.py", line 82, in copyfileobj
fdst.write(buf)
OSError: [Errno 28] No space left on device

R: 0 / I: 0 / P: 13

I'm posting this so you guys won't create conspiracy theories. I know, my hope us futile.
If everything goes well soon the site's and our problems will be resolved. But they will turn the site read-only mode for a while. I have no information how long the process will take. Keep calm and be patient.

Thanks.

R: 0 / I: 0 / P: 13

Release Tomorrow!

I had a difficult week, but I got some good work done. As well as some fixes and improvements to the new 'has audio' detector and a bunch of other bug fixes, there is a neat new way to input logically complicated tag searches, and I reshaped the repository processing pipeline to no longer block the program--you can now keep browsing your files while repository work is going on!

The release should be as normal tomorrow. I have some messages to catch up on, which I will try to clear out tomorrow.
Open

R: 1 / I: 0 / P: 13

Database Encrypted Sync

I haven't seen this discussed so I thought I'd make a thread on it.

We are all aware that Hydrus runs fine inside a Truecrypt container, but having one huge file can be a problem.

For instance if you backup your 500GB database inside a truecrypt container and then download even just 1 image you basically need to backup the whole 500GB all over again if you run an automated backup to an external HDD or a NAS. Another problem with Truecrypt/Veracrypt volumes is you have to set the container size at the time of making it, leaving it too big or too small all too easily.

Well I've been experimenting with different crypto programs that feature encrypting directories while keeping files separate. The filenames and contents are unreadable, but they are still their own files so syncing and backup programs know exactly what to copy and what is the same.

The problem is that Hydrus doesn't want to run at all within these encrypted directories, but luckily we can have our installed client separate to the db folder.
>install an encryption program (I used cppcryptfs but there are a bunch for various platforms)
>create a new encrypted folder on any drive
>mount it as a Drive letter (you will need to use the same Drive letter each time)
>Move your Hydrus db folder to that mounted drive (or you can make a new one by ignoring this step)
>create a shortcut to your client.exe adding -d="path to db folder on virtual drive" (e.g "C:\Hydrus Network\client.exe" -d="Z:\db")
>use shortcut

it should find your database and start like normal, with any new files being encrypted as their own files that can be synced individually.

I haven't tried it but theoretically you could even mount a cloud provider as a virtual drive and store your entire database that way, probably stupid but I might try it for fun.

Old thread: http://archive.fo/ybNay
Open

R: 12 / I: 0 / P: 13

Version 363

https://youtube.com/watch?v=VWY8rYPuftY [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v363/Hydrus.Network.363.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v363/Hydrus.Network.363.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v363/Hydrus.Network.363.-.OS.X.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v363/Hydrus.Network.363.-.Linux.-.Executable.tar.gz
source
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v363.tar.gz

I had a good week of work. There is new 'has audio' detection for videos, extensions to the client api, and plenty of bug fixes.

has audio

The client can now detect if a video has an audio channel and, if so, presents that information with the unicode character ๐Ÿ”Š on its thumbnail-right click and in the media viewer! Audio files and swf files will get this as well. You can now also search for this status (or exclude it) with the new 'system:has audio' search predicate.

This will apply to all new files. All existing video files will need to be reparsed to see if they have audio. This is a large job and so has been queued up in the new files maintenance system (under database->maintain->review scheduled file maintenance). By default, I have this system throttled to 100 files/day in idle time, but I suspect I will have to improve this throttle to do work more frequently in smaller batches, since if you have 50,000 video files, that's a lot of time at 100 files/day to catch up! You can of course manually run this job from the maintenance panel, although bear in mind it likely totals several hours of heavy CPU work to finish.

The new system actually reads the file's audio and tries to determine if it is one of those 'fake' silent channels. If you discover any false-positives here, where the video is silent but the system still thinks there is audio, or indeed false-negatives, please send some examples to me so I can check them out.

You can change the speaker unicode character under the new options->sound panel (for instance, if it does not display correctly on your machine), and increase the file maintenance throttle under options->maintenance and processing.

client api

The Client API gets some extensions this week. The a file's 'known_urls' and the new 'has_audio' status is now sent on a file_metadata request, and I have added some simple cookie management.

The creator of the Hydrus Companion, a browser add-on that talks to the Client API, has already written support for this cookie management! It allows you to copy your cookies from your web browser to hydrus, letting it 'log in' to sites the current downloader system cannot handle. Please check it out here:

https://gitgud.io/prkc/hydrus-companion


the rest

'system:hash' and 'system:similar to' now support multiple hashes. This lets you do larger lookups, if, say, you have 100 md5s. Also, the thumbnail right-click->file relationships->find similar files will now work on a selection of multiple thumbnails.

The file maintenance manager panel now has some text to explain what to do and a new 'see description' button to say more about what each job does.

Subscriptions will now 'publish' their file results to pages/popups as each query finishes. This is a small thing, but it makes for a slightly smoother stream of new files if you are eagerly watching a large sub take takes a while to sync.

The tag autocomplete inputs should be a bit more receptive and accurate to fast typing and entering of tags. Let me know if you still have problems!

R: 0 / I: 0 / P: 13

never give up

Top