- fixes and misc:
- fixed a problem that made saved page file sorts reset their sort order one time on update to v492. thank you to a user for noticing this and discovering the fix, and I'm very sorry for the inconvenience of changing your session and favourite search sorts. unfortunately there is no easy fix other than rolling back to a backup and jumping forward to this version
- fixed a v492 message display error when setting various duplicate relationships to three or more thumbnails at once. it was a stupid typo, sorry for the trouble! (issue #1199)
- if a page tab name elides to a 'shorter...' length, it now has its full name as the tooltip
- fixed a typo in update code error handling (issue #1192)
- the duplicate filter page now remembers if you are 'searching immediately'/'search paused' (issue #1193)
- if you are on non-Windows and export files manually or with an export folder to an NTFS or exFAT partition, this is now detected, and NTFS-invalid characters in the pattern-generated folders or filename are now replaced with underscores (issue #1194)
- 'fixed' a system predicate bug in the 'OR*' advanced predicate parser--entering a logical expression that results in a negated system tag now causes an error. previously, it would strip the 'system:' and just enter the given text as an unnamespaced tag. furthermore, that dialog now reports specific error reasons when it fails to parse. I hope to improve support for negated system tags in future--some stuff, like archive/inbox, should be easy.
- I think I fixed an instance where the archive/delete filter's confirmation dialog could present 'delete from hard disk' as an option when it wasn't appropriate
- in an attempt to reduce the media-change flickering we've recently seen in the media viewer, I untangled a bunch of the canvas size/position code this week. I'm preparing a complete overhaul and neat Qt layout integration, which this starts. I _think_ I've made some things less flickery on occasion, but we'll see IRL. much more to do
- added a '--profile_mode' launch argument, which allows you to capture the performance of boot and also try out profile mode on the server (although support there is very limited atm)

next week

Next week is a medium size job week. I want to put some time into note parsing. I am not sure how far I can get, but fingers crossed I can actually get 'note import options' and a note content parser working, and we'll be able to update the existing downloaders to grab artist notes and things.

Don't know what I'm supposed to do to fix this or if it was broken before.
Hydrus v464, first time I've opened it in a few months and says you want feedback.

>>1327
Ah, damn, I think that login script does not work any more, I think I remember another user saying this recently. If any script suddenly breaks, I recommend you disable the login (set it to 'active: no' under manage logins) and use Hydrus Companion instead, which will sync your cookies over from your logged-in browser and works a lot better for all unusual situations:

https://gitgud.io/prkc/hydrus-companion

I will see about removing that rule34.xxx link in the defaults so new users don't get hit by this either.

Is there a way to change the upper limit on gifs from 32?
I'm trying to save a 52Mb sized gif and I got prompted with this message.

I had a great week with two big changes. First, there are Qt6 test builds for advanced users to try out, and second, in prep for a soon-to-come Note Import Options, I added filetype filtering and 'use default at time of import' tech to File Import Options.

The release should be as normal tomorrow.

>>1331
Sure, check the 'File Import Options' of the importer you are working on. There's a setting in there just for gif size. options->importing for your default settings.

>>1329
fuck me this was meant for the Hydrus thread lol

for anyone who comes across this thread here's how i fixed it
>import file
>click file import options
>click on the drop down and select the "set custom file import options"
>change maximum gif filesize or check no limit
>click apply
hopefully this helps someone

It would be great to allow unrestricted file extensions.
I want to import into Hydrus many 3D models (.stl; .step; .prt; .x_t; .obj; et cetera) and some others like .txt and .html.
Viewers for non supported files are not necessary if a custom screenshot (provided by the user) of the file content is showed in Hydrus.
Then the workflow would be to import, to tag, to add a screenshot, to search, and then to execute with an external program.

It would be great to allow unrestricted file extensions.
I want to import into Hydrus many 3D models (.stl; .step; .prt; .x_t; .obj; et cetera) and some others like .txt and .html.
Viewers for non supported files are not necessary if a custom screenshot (provided by the user) of the file content is showed in Hydrus.
Then the workflow would be to import, to tag, to add a screenshot, to search, and then to execute with an external program; or to export and then to execute.

I have three suggestions:

I wish there was some kind of favorites functionality. Nothing fancy, just a system tag for favorites, a quick-add button for said tag the right-click menu, and maybe a star on the thumbnails.

The current workflow for importing large queries (artists) is shit. Subscriptions aren't for large imports, but if you want to grab an artists stuff you're going to need one. So you have to manually do a run with the importer and manually make an identical subscription. I'd like to be able to right-click on an import query and pick an option to just make it a subscription.

It would be nice if there was a way or preserve generic metadata. I am thinking specifically of descriptions on pages like FA or e621. Nothing huge, just text box basically where importers could dump whatever into.

>>997
And one more thing! Currently when you import things they get their modified and imported dates set to when they're downloaded. Usually that means it's backwards slightly out of order from chronological. Subsequent imports will have the same issue but also all be 'after' that. Effectively there's no way to sort by chronological order. I want items had a 'posted date' (that importers would need to parse) that you could sort on.

Hi,

I am curious to know your thoughts on my proposal below.

I welcome your feedback or rejection ( I pray not : - ) )

======Original message=======


Hey,

The new normal seems to be daunting but it's high time for us to shift our focus on innovative ways to start a business.

I noticed that you shared https://mode.com/blog/python-interactive-plot-libraries/.

i'd like to offer http://endchan.net/hydrus/res/25.html an https://www.guru99.com/ article that may help a start-up business take off in these challenging times.

- misc:
- using black magic, I have made it so the shortcuts for 'move left/right one page' 'and 'move home/end' do not dip down to the lowest level of a neighbouring page of pages for the next command. it now stays on the current tab level for three seconds after the most recent move command. this works in testing but may be jank in some IRL situations, so if this matters to you, let me know how it works out
- fixed a bug in 'do a full metadata resync' that meant unprocessed row orphans were not being deleted, which lead to lingering 1950/2000-style processed gauges that didn't actually cause any work to be done on 'process now'
- the duplicate filter now shows if one or both files have an icc profile. for now the score for this is always 0, neutral
- I think I have reduced general lag on some busy clients
- .
- code cleaning and minor fixes:
- refactored file viewing stats management to a new database module
- refactored file physical storage management to a new database module
- cleaned up an ugly bridge that made inbox/archive work and moved it all to a clean new separate database module
- improved some client file physical storage repair code, both in how it repairs and how it recovers in the current boot
- updated the yes/no dialog texts when you apply 'not related' or 'alternates' to a selection
- added a bunch of tooltips to the 'speed and memory' options panel. also clarified the example image sizes in number of pixels
- improved how my grid layout propagates tooltips from the widget to the text when the widget is compound and in its own layout
- consolidated where the delete lock test occurs to just one location for db, gui
- added infrastructure to filter and report delete-locked files. callers no longer care about specific lock rules, opening this up to future expansion
- cleaned and simplified some duplicate action processing code

full list

- system predicates:
- the advanced OR input, where you can type tags in complicated logical expressions, now supports system predicates! most system predicates are supported using their typical display strings. it uses the same engine as the client api, so check the examples here https://hydrusnetwork.github.io/hydrus/developer_api.html#get_files_search_files sorry for the delay here
- the advanced input also runs tags better through the hydrus tag 'cleaning' process, so things like whitespace between the namespace colon and the subtag are cleaned up correctly, and invalid tags should be excluded
- it also starts with the keyboard focus in the text input
- and I think I fixed an issue with '!'', 'not', or '-' negation prefixes not parsing
- highlighted the example parseable system predicate texts in the Client API help, and added 'last viewed' to it
- .
- misc:
- altering your services in _manage services_ no longer causes a full page refresh for all currently open search pages
- in a related thing, if you click the file or tag domain of a file search page to be the same as it just was, you no longer get a page refresh
- the rating widgets now show their current rating value on their tooltips
- when setting a numerical rating by a drag, it no longer matters if your mouse strays above or below the widget--it will still set
- the String Processing system has a new 'String Tag Filter' processing step. this applies the normal tag filtering object to your list of strings and also performs the hydrus 'tag cleaning' process on them, making them all lowercase and trimming whitespace and so on
- the sibling/parent sync is now even more polite when told to do work in 'normal' time. this has been hitting a lot of new users really hard, so it should now really trickle work during normal time, throttling down when it hits a bump to avoid stunlocking you but also responding quickly to recent changes if you are fully synced
- the database repair code is now better at healing damaged fast-text-search (FTS) tables. previously, in cases of partial damage to the virtual table, the repair code would error out

- new import/export framework:
- started a new modular metadata import/export pipeline. this thing starts out today by doing the work of newline-separated tags in a .txt sidecar file and will expand to do all sorts of metadata in other formats like JSON and XML. it will also, eventually, support arbitrary cross-type conversions like tags to urls or ratings to tags
- export folders now support '.txt' sidecar tag exporting!
- the '.txt' sidecar tag importing in import folders or manual imports is now handled by the new pipeline
- the '.txt' sidecar exporting in the manual export dialog is now handled by the new pipeline
- please expect the UI around '.txt' sidecar importing and exporting to change significantly in future. you'll be selecting different metadata types to import or export, make string processing steps to alter or filter what you get, and of course be able to compile it all into more complicated filetypes
- .
- cleanup and refactoring:
- mr bones gets two new columns to line up the numbers better
- a bunch of export code got moved around. created a new module 'exporting', and moved ClientExporting.py to it, renaming to ClientExportingFiles.py
- removed an old prototype for sidecar exporting and related plans for UI
- the 'missing file folders on boot' dialog now points users to 'help my media files are broke.txt'
- brushed up the 'help my x is broke.txt' documents in the database directory a little
- fixed some surplus double backslashes in the help
- a secret tiny label change/fix, let's see if anyone notices
- cleaned up how the rating widgets manage and update rating state. it was ancient bad code
- updated how different rating values are converted to UI text

I had a good week. I did a mix of background cleanup along with quality of life and other improvements, mostly for advanced users.

The release should be as normal tomorrow.

- orphan files:
- there's a persistent processing bug some users have where some update files are missing but they won't redownload correctly. I think I fix that this week naturally so existing maintenance routines will now be able to fix it themselves after another round
- fixed some issues related to deleting files from the repository updates file domain.
- the 'clear orphan file records' maintenance command now fixes the 'all my files' umbrella services as well as the 'all local files' one. it also has nicer description, does some additional file-removal cleanup, and triggers a file recount if problems are found
- moved 'clear orphan files' to the 'files' maintenance menu

next week

Next week is a medium size job week. I want to have another go at building a larger metadata import/export pipeline. I want to start unifying tags, urls, ratings, everything into one thing that can eat up or spit out XML or JSON.

I was ill this week and am short on work time. I will spend tomorrow doing some more normal work instead of the release.

491 should be on the 13th of July. Thanks everyone!

I had a good couple of weeks. I overhauled an ancient system behind the scenes and did a heap of little jobs, fixing a bunch of bugs and improving quality of life.

The release should be as normal tomorrow.

- archive/delete:
- the final 'commit/forget/back' confirmation dialog on the archive/delete filter now lists all the possible local file domains you could delete from with separate file counts and 'commit' buttons, including 'all my files' if there are multiple, defaulting to the parent page's location at the top of the list. this let's you do a 'yes, purge all these from everywhere' delete or a 'no, just from here' delete as needed and generally makes what is going on more visible
- fixed archive/delete commit for users with the 'archived file delete lock' turned on
- .
- misc:
- fixed a bug in the parsing sanity check that makes sure bad 'last modified' timestamps are not added. some ~1970-01-01 results were slipping through. on update, all modified dates within a week of this epoch will be retroactively removed
- the 'connection' panel in the options now lets you configure how many times a network request can retry connections and requests. the logic behind these values is improved, too--network jobs now count connection and request errors separately
- optimised the master tag update routine when you petition tags
- the Client API help for /add_tags/add_tags now clarifies that deleting a tag that does not exist _will_ make a change--it makes a deletion record
- thanks to a user, the 'getting started with files' help has had a pass
- I looked into memory bloat some users are seeing after media viewer use, but I couldn't reproduce it locally. I am now making a plan to finally integrate a memory profiler and add some memory debug UI so we can better see what is going on when a couple gigs suddenly appear
- .
- important repository processing fixes:
- I've been trying to chase down a persistent processing bug some users got, where no matter what resyncs or checks they do, a content update seems to be cast as a definition update. fingers crossed, I have finally fixed it this week. it turns out there was a bug near my 'is this a definition or a content update?' check that is used for auto-repair maintenance here (long story short, ffmpeg was false-positive discovering mpegs in json). whatever the case, I have scheduled all users for a repository update file metadata check, so with luck anyone with a bad record will be fixed automatically in the background within a few hours of background work. anyone who encounters this problem in future should be fixed by the automatic repair too. thank you very much to the patient users who sent in reports about this and worked with me to figure this out. please try processing again, and let me know if you still have any issues
- I also cleaned some of the maintenance code, and made it more aggressive, so 'do a full metadata resync' is now be even more uncompromising
- also, the repository updates file service gets a bit of cleanup. it seems some ghost files have snuck in there over time, and today their records are corrected. the bug that let this happen in the first place is also fixed
- there remains an issue where some users' clients have tried to hit the PTR with 404ing update file hashes. I am still investigating this

I had a good week. I did a variety of small work and one important bug fix that should speed up media browsing for power users.

The release should be as normal tomorrow.

I had a good week. I did a mix of cleanup and improvements to UI and an important bug fix for users who have had trouble syncing to the PTR.

The release should be as normal tomorrow.

full list

- misc:
- updated the duplicate filter 'show next pair' logic again, mostly simplification and merging of decision making. it _should_ be even more resistant to weird problems at the end of batches, particularly if you have deleted files manually
- a new button on the duplicate filter right hover window now appends the current pair to the parent duplicate media page (for if you want to do more processing to them later)
- if you manually delete a file in the duplicate filter, if that file appears again in the current batch of pairs, those will be auto-skipped
- if you manually delete a file in the duplicate filter, the actual delete is now deferred to when you commit the batch! it also undoes if you go back!
- fixed a bug when editing the external program launch paths in the options
- fixed an annoying delay-and-error-popup when clearing the separator field when editing a String Splitter. now the field just turns red and vetoes an OK with a nicer error text
- also improved how string splitters report actual split errors
- if you are in advanced mode, the _review services_ panels now have an 'id' button that lets you fetch the database service id
- wrote a new database maintenance routine under _database->check and repair->resync tag mappings cache files_, which is a lightweight way of fixing ghost files or situations where files with a tag are neither counted nor appear in file results. this fixes these problems in a couple minutes, so for this it is much better than a full regen of the cache
- .
- cleanup and other boring stuff:
- the archive/delete filter now says which file domain it will be deleting from
- if an archive/delete filter is launched on a 'multiple locations' file domain, it is now careful to only make delete records for the deleted files for the file services each one is actually in
- renamed the 'default local file search location' option to 'fallback' and updated its tooltip a bit. this was really a hacky thing I needed to fill some gaps while rewriting from 'my files' to multiple local file services. the whole thing needs more attention to become more useful. I also fixed an issue where it could become invalid 'nothing' if you deleted a file service it was referring to (issue #1155)

I had a good week. I did some simple work to make a clean release before my vacation.

The release should be as normal tomorrow.