/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Posting mode: Reply

Check to confirm you're not a robot
Name
Email
Subject
Comment
Password
Drawing x size canvas
File(s)

Board Rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Manage Board | Moderate Thread

Return | Magrathea | Catalog | Bottom

Expand All Images


Version 462 Anonymous Board owner 11/17/2021 (Wed) 22:40:54 Id: e7725a [Preview] No. 1169
https://youtube.com/watch?v=Gu2Iayi3AvM [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v462/Hydrus.Network.462.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v462/Hydrus.Network.462.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v462/Hydrus.Network.462.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v462/Hydrus.Network.462.-.Linux.-.Executable.tar.gz

I had a good week with a variety of fixes and optimisations. New users also get a couple of user-friendly services to start with.

highlights

A recent update to the mpv seekbar accidentally caused bad errors during audio playback. I have now fixed it. I regret the regression, and I have added tests to make sure this does not happen again.

All 'file log' import queues now search and count themselves faster. Any heavy import session, like a very large downloader or many medium size downloaders working at once, should use less CPU just to do normal work.

I sped up some file searches that were running super slow. The general problem file search was typically a mix of 'has multiple search predicates' and 'includes a short unnamespaced tag or namespace:anything'. This combination was taking two minutes or more rather than a handful of seconds due to an overly complicated database query that wasn't running well. Should be fixed now, but let me know if you have any more of this sort of trouble!

new user services

To help new users, new clients now start with two extra services:

First is a local tag service called 'downloader tags', which the initial default tag import options is now set to get tags for. I hope this will reduce some of the confusion around default tag import options, merged downloader/personal tags in 'my tags', and having to retroactively fill in missed tags for files downloaded early on.

Second is a like/dislike rating service called 'favourites'. Many users are unaware of the existence of ratings services at all, so now everyone will start with a simple clickable star. When you update, if you currently have no rating services, you will be asked if you want to get this new default. You don't have to, but if you haven't tried ratings out yet, give it a go!

full list

- misc:
- fixed a recent serious regression that could cause a crash when playing audio in mpv (issue #1007)
- the main importer file log now does 'get next/all/count imports with status y' calls significantly faster, particularly on very large lists. these calls happen all the time for different status text changes and general 'which import to try next?' work. all busy downloader situations should see CPU gains to regular and background work
- fixed a problem where importing with the min/max file resolution options set would give a typo error when the size was violated rather than 'ignored'
- I think I have fixed an issue with subscriptions not wanting to run a query if by random accident that query has an invalid URL selected as the query's 'example url' for various pre-work login and bandwidth tests
- hydrus can now capture duration/fps of videos that specify two very close fps, e.g. 60 and 59.99. previously, these would default to the 24 fallback and could cause some weirdness in mpv
- replaced the default pixiv artist page api parser with one that fetches the newer url format, matching the tag search. existing users will see no automatic change but will receive the new parser, so if you are a big pixiv user, you might like to switch 'pixiv artist gallery page api' to the 'new urls' parser variant under _network->downloader components->manage url class links_. note that if you do this, your pixiv artist subscriptions will do a mini-resync (which involves some wasted time/bandwidth) as their urls change!


Anonymous Board owner 11/17/2021 (Wed) 22:41:23 Id: e7725a [Preview] No.1170 del
- network redirect improvements:
- gallery page jobs now give their child 'next gallery page' jobs themselves as a referrer
- when the gallery downloader gets a 3XX redirect, the file import objects and next gallery pages it makes now get the redirected URL as referral url (previously, it used the original gallery url)
- when the post downloader gets a 3XX redirect, the redirected url is now added as a primary source url
- when the post downloader gets a 3XX redirect, child import objects and direct file downloads now get the redirect URL as referral url (previously, it used the original post url)
- when the raw file downloader gets a 3XX redirect, the redirected url is now added as a primary source url
- when the raw file downloader gets a 3XX redirect to a Post URL, it now tries to queue that URL back up in the file log, just like when a gallery fetch comes back with a Post URL. some safety code stops potential loops here as well
- .
- new services:
- a new client now starts with a second local tag service called 'downloader tags'. default tag import options are now initialised in a fresh client to pull all file post tags to this service. this should relieve new users' confusion about setting up default tag import options
- similarly, a new client now starts with a like/dislike rating service called 'favourites'. existing users who have no rating services will be asked if they want to get it on update. many users are unaware of the rating system, so this is a chance to play with it
- the 'getting started with downloading' 'and '...with ratings' help has some updated screenshots and talk about the new default services and parsing
- .
- database fixes:
- fixed a very slow database query that was involved with file search using unnamespaced tags when other search predicates had already limited the scope of search
- fixed a similar slow query when the 'bad' search predicate was 'namespace:anything', particularly when the namespace is a wildcard
- fixed the 'clear orphan tables' database maintenance routine. it had decayed due to bit rot and was deleting new repository processing tracking tables. the routine is now plugged directly into the new database modules system, and any module now can be upgraded to discover surplus service tables. the system has been updated to permit the detection and removal of duplicate tables in the wrong database file, and it also makes a statement if no orphan tables were found
- the 'metadata resync' repository maintenance task now removes surplus file rows from the processing tracking tables
- the 'metadata resync' repository maintenance task now corrects content type rows in the main processing tracking table
- the process of registering updates to a repository is now a little faster, more reliable, repairs basic damage, and keeps more good work when damage needs to be repaired
- I _think_ the users who were still getting PTR processing errors related to a database confusion about content/definitions update files should now be fixed after another full metadata resync! please let me know if there are still problems

next week

This week ended up being packed, so I did not have time for larger stuff like multiple local file services. I'll try again to move it forward.


Release Tomorrow! Anonymous Board owner 11/24/2021 (Wed) 06:38:50 Id: e01736 [Preview] No.1173 del
I had a great week. As well as some little fixes and quality of life work, I improved support for ogv files (oggs with video), and completely reworked the 'presentation' options of importers, making the whole system faster and more user-friendly.

The release should be as normal tomorrow.



Top | Catalog | Post a reply | Magrathea | Return