/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Posting mode: Reply

Check to confirm you're not a robot
Drawing x size canvas

Remember to follow the rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Manage Board | Moderate Thread

Return | Catalog | Bottom

Expand All Images

Version 491 Anonymous Board owner 07/13/2022 (Wed) 22:23:54 Id: 1973fd [Preview] No. 1316
https://youtube.com/watch?v=OQEDWiM-QRI [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v491/Hydrus.Network.491.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v491/Hydrus.Network.491.-.Windows.-.Installer.exe
app: https://github.com/hydrusnetwork/hydrus/releases/download/v491/Hydrus.Network.491.-.macOS.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v491/Hydrus.Network.491.-.Linux.-.Executable.tar.gz

I had a good couple of weeks doing a behind the scenes overhaul and a variety of quality of life work. I was ill last week and put off the release, but I am feeling great now.

metadata import/export

As is often the case, an important overhaul makes few actual changes front end. I've been trying to do this for a while, but now the 'export/import tags in a neighbouring .txt file' routine now works on completely new tech. Rather than hardcoded tags into .txt files, it now uses a modular system that I will be able to expand in future to support filtering and string processing, more metadata types like ratings and URLs, JSON and XML as the file formats, and will even allow funky migrations like converting tags to URLs.

As a side benefit, Export Folders now support tags .txt export.

For now, all the UI front-end looks the same, but please expect this to change in future. I'll be writing some nice unified panels and dialogs to handle the new objects as I write them.

advanced user highlights

The 'OR*' advanced tag input now supports system predicates! It uses the same system predicate parser as the Client API, so you can now type or copy/paste most system predicate text and get something both useful and complicated. The text that parses isn't always exactly the same as the predicate label, so check out the big list of example parseable system predicates here: https://hydrusnetwork.github.io/hydrus/developer_api.html#get_files_search_files

Also, if you are a parser creator, String Processing now has a Tag Filter processing step. Let's say you can grab all the tags from somewhere but you need to filter out a handful of non-tag text like '+' and '?', or you are able to create hydrus namespaced tags and want to filter by namespace, just insert this into your string processing and it should be much easier than messing around with long regexes.

Anonymous Board owner 07/13/2022 (Wed) 22:25:18 Id: 1973fd [Preview] No.1317 del
full list

- system predicates:
- the advanced OR input, where you can type tags in complicated logical expressions, now supports system predicates! most system predicates are supported using their typical display strings. it uses the same engine as the client api, so check the examples here https://hydrusnetwork.github.io/hydrus/developer_api.html#get_files_search_files sorry for the delay here
- the advanced input also runs tags better through the hydrus tag 'cleaning' process, so things like whitespace between the namespace colon and the subtag are cleaned up correctly, and invalid tags should be excluded
- it also starts with the keyboard focus in the text input
- and I think I fixed an issue with '!'', 'not', or '-' negation prefixes not parsing
- highlighted the example parseable system predicate texts in the Client API help, and added 'last viewed' to it
- .
- misc:
- altering your services in _manage services_ no longer causes a full page refresh for all currently open search pages
- in a related thing, if you click the file or tag domain of a file search page to be the same as it just was, you no longer get a page refresh
- the rating widgets now show their current rating value on their tooltips
- when setting a numerical rating by a drag, it no longer matters if your mouse strays above or below the widget--it will still set
- the String Processing system has a new 'String Tag Filter' processing step. this applies the normal tag filtering object to your list of strings and also performs the hydrus 'tag cleaning' process on them, making them all lowercase and trimming whitespace and so on
- the sibling/parent sync is now even more polite when told to do work in 'normal' time. this has been hitting a lot of new users really hard, so it should now really trickle work during normal time, throttling down when it hits a bump to avoid stunlocking you but also responding quickly to recent changes if you are fully synced
- the database repair code is now better at healing damaged fast-text-search (FTS) tables. previously, in cases of partial damage to the virtual table, the repair code would error out
- fixed a bug where certain search predicate calendar dates that are acceptable in Linux but not in Windows caused Windows to fail to load the session. if you put in 1965 as a search date, it should now revert to the current time one next load etc...
- the test to see if a directory is writeable-to is improved and now handles Windows's Program Files directory correctly
- improved how the boot scripts handle incorrect/bad database directory paths. the error handling works better, and it figures out a fallback location for crash.log better
- a new button on 'review services' now lets advanced users copy the service key to the clipboard
- the migrate tags dialog now lists file repositories, ipfs services, and 'all my files' as potential file filter domains
- when checking it has space for a large transaction like a vacuum, hydrus now tries to check if you are running on a ramdisk or other severely space-limited temp dir and offers more text if this is true
- updated the '4chan style thread api parser' to handle posts with multiple files, which fixes tvchan.moe and probably anything else running NPFchan
- some logic testing around showing 'return to inbox' and the actual operation is fixed so it only applies to local files. in some weird advanced situations, you could previously send deleted files to inbox

Anonymous Board owner 07/13/2022 (Wed) 22:25:50 Id: 1973fd [Preview] No.1318 del
- new import/export framework:
- started a new modular metadata import/export pipeline. this thing starts out today by doing the work of newline-separated tags in a .txt sidecar file and will expand to do all sorts of metadata in other formats like JSON and XML. it will also, eventually, support arbitrary cross-type conversions like tags to urls or ratings to tags
- export folders now support '.txt' sidecar tag exporting!
- the '.txt' sidecar tag importing in import folders or manual imports is now handled by the new pipeline
- the '.txt' sidecar exporting in the manual export dialog is now handled by the new pipeline
- please expect the UI around '.txt' sidecar importing and exporting to change significantly in future. you'll be selecting different metadata types to import or export, make string processing steps to alter or filter what you get, and of course be able to compile it all into more complicated filetypes
- .
- cleanup and refactoring:
- mr bones gets two new columns to line up the numbers better
- a bunch of export code got moved around. created a new module 'exporting', and moved ClientExporting.py to it, renaming to ClientExportingFiles.py
- removed an old prototype for sidecar exporting and related plans for UI
- the 'missing file folders on boot' dialog now points users to 'help my media files are broke.txt'
- brushed up the 'help my x is broke.txt' documents in the database directory a little
- fixed some surplus double backslashes in the help
- a secret tiny label change/fix, let's see if anyone notices
- cleaned up how the rating widgets manage and update rating state. it was ancient bad code
- updated how different rating values are converted to UI text
- misc cleanup of some free space checking code
- fixed some bad quote characters in client api help JSON examples
- improved some error handling for uploading pending content and sped up file uploads a little

next week

Next week is a cleanup week. I'll try and break up some more monolithic database code.

Release Tomorrow! Anonymous Board owner 07/20/2022 (Wed) 01:31:45 Id: 1003db [Preview] No.1320 del
I had a good week. I did a mix of background cleanup along with quality of life and other improvements, mostly for advanced users.

The release should be as normal tomorrow.

Top | Return | Catalog | Post a reply