test

I managed to get a first version of an increment/decrement integer rating service working, and I'm happy with it, but it needs more polish and my day tomorrow is crazy, so I'm putting off the release a week to finish it up and get more things done.

v519 should be on the 8th of March. Thanks everyone!

I had a good couple of weeks. There's some small bug fixes and quality of life, improved server janitor tools, and a new 'rating' service that lets you count with clicks.

The release should be as normal tomorrow.

I had an ok week. There's a mix of small fixes, the addition of hours:minutes to system:time calendar predicates, and a neat overhaul of how autocomplete tag search presents its pre-fetch results on slow searches.

The release should be as normal tomorrow.

I had another light week. There's some misc fixes, symlinks for export folders (thanks to a user!), a new way to show image transparency (which will particularly help in the duplicate filter), and I finally removed superfluous sibling and parent suggestions from 'related tags'.

The release should be as normal tomorrow.

I had another light week. I added some tuning options for the new 'related tags' search and cleared a handful of misc small work.

The release should be as normal tomorrow.

related tags

EDIT: In final testing, this stuff was working slow IRL for me, so I reduced its search space for today. You are going to see a lot of 'search domain is too large' on the PTR. I'll keep working!

Thanks to a user who sent in a great proof of concept, the 'related tags' column (which shows in manage tags and you can turn on under options->tag suggestions) has some new buttons to test. I called them 'new 1' and 'new 2'. The first works fast but may have limited results on repositories, the second--which only shows on repositories--can only search small-count tags but gives deep results.

Secondly, if you select some tags in the main tags list, the 'related tags' search will now only search those tags! If you can't remember a character name, then select a 'series' tag and click for a related tags search, and you'll get a bunch of them!

Thirdly, these new search buttons now work on manage tags launched from multiple thumbnails!

Please give this a go and let me know how you get on with IRL data. The quality of the new search results is better than my old routines, but I need to do a bunch more tuning and optimise their search. Are there too many recommendations (is the tolerance too high?)? Is the sorting good (is the stuff at the top relevant or often just noise?)? Which source tags (e.g. 'page:x') give the worst suggestions?

I'm sufficiently happy with the new performance, particularly with the tag selection, that once this is working nice, I think I'll make it default-on for everyone. Lots of people missed this system, and I always thought it was a bit jank.

other highlights

I wrote some sidecar import/export help here! https://hydrusnetwork.github.io/hydrus/advanced_sidecars.html

>>1453
twitter gay, thanks for fix
I suppose this means my nitter subs are broken too?

On a different topic, does Hydrus do anything with DeepBooru or are there any extensions for it you know of that I can maybe use to try some autotagging?

>>1455
Is DeepBooru the same as Deep Danbooru, or the same basic model, so you could switch it in place? Hydrus-dd offers several ways to plug into Deep Danbooru, here:

https://gitgud.io/koto/hydrus-dd

I was talking with someone else today about how I'd like to do more with this sort of tech in future, just making it easier to work with the sidecars you need or the Client API whatever. I'm sure image-to-text is going to benefit from all this text-to-image tech that's now floating around, especially now it is becoming reasonable to train your own models to do stuff. Hydrus as a big metadata-rich archive seems like it plugs into that tech pretty well!

I unfortunately had a light week. The new 'related tags' search is polished and turned on by default for all users, and I fixed some small bugs.

The release should be as normal tomorrow.

>>1448
>Minis Forum
I recently picked up an Minis Forum HM80, and I like it a lot. It's my first computer from this company, and it's my first desktop in about a decade. I've been using laptops for so long before this, but this is a nice change of pace. It's a lot faster than I thought it would be too.

I had a low productivity week, so I am going to skip the release and get more done on Wednesday.

v514 should be on the 25th of January. Thanks everyone!

----

For Client API developers, I am clearing out some obsolete calls and data structures in v514! Since we have a spare week, please review the pending breaking changes:

* '/add_tags/get_tag_services' is removed! use '/get_services' instead!
* 'hide_service_names_tags', previously made default true, is removed and its data structure 'service_names_to_statuses_to_display_tags' is also gone! move to the new 'tags' structure.
* 'hide_service_keys_tags' is now default true. it will be removed in 4 weeks or so. same deal as with 'service_names_to_statuses_to_display_tags'--move to 'tags'
* 'system_inbox' and 'system_archive' are removed from '/get_files/search_files'! just use 'system:inbox/archive' in the tags list

Full documentation preview, including some new commands and parameters, here:

https://hydrusnetwork.github.io/hydrus/developer_api_future.html

>>1450
>Yeah I almost feel like some part of them fell off the back of a truck,
The price is great. It's very affordable, and you get a lot of bang for your buck.

I had a good couple of weeks working on a lot of different things. There's updated downloaders, a heap of bug fixes, optimisation and quality of life, a significant Client API update, and a new related tags method to test out.

The release should be as normal tomorrow. (actually later today)

I had a good week doing one thing. The Client API now supports the main duplicates commands, fetching current file relationships and the main filter workflows, and then setting basic file relationships back again. This is a complicated feature for users who play around with the latest Client API tech, so tomorrow's release is for those advanced users only!

The release should be as normal tomorrow.

Is there a way to show the total duration of all selected video files? It shows the total file size (rounded), but I can't find an option to show the total duration.

Is there some way to apply to all selected images tags like page:1, page:2, etc in same order as it visible now?

>>1442
I had this exact same thought over my vacation when I was looking at some vidya captures I had made. I'll add this!

>>1443
Not yet, but I would like to add this. I've never really been happy with how hydrus handles 'page:1' style tags, but we'll never get better at it as long as it is difficult to easily add them to files after they are imported.

I had a great week back at work. I cleared a variety of small jobs and spent some extra time on the duplicates system. Several duplicates search bugs are fixed, and you can now set a potential pairs search for 'one file is in search A, the other is in search B'.

The release should be as normal tomorrow.

I had a good week mostly fixing bugs, including the recent annoying-but-harmless shutdown bug some users had and the macOS app boot bug. There's also some UI quality of life.

The release should be as normal tomorrow.

>>1425

Thanks for your continuing work on the project man, much appreciated. I have some feedback/wishes:

1. 4chan thread watcher via Tor gives 403 Cloudflare error - simple downloader works. Is there something one can do about this beyond switching circuits/proxy/clearnet? I mentioned this before and the browser header shouldn't make a difference here, as downloads as such work, just not the watcher.

2. 4chan simple downloader fails at .webms in same setup, these have to be manually added.

3. Add a second/third bar for tabs, so that the titles are not squeezed when having many tabs open.

4. PDF/ebook compatbility to use Hydrus for more informative tasks (this might get you a lot of support from librarian types).

5. Grouping of tags - as with the namespace/creator group by function, have colored bars or something that groups all items belonging to that category - or some similar function to have a "folder like" grouping in the main window. Tags are awesome for precise searches/classifications, but sometimes that "all in one" feeling that folders offer might be useful; I may have overlooked something though.

As always, I really love the work you do and you simplify a lot of time intensive stuff. I'd love to have a setup working that scrapes threads automatically by topic, I was thinking of a hydrus + something scripted setup that identifies threads by title/keyword and then automatically adds them to the watcher. This would save so much time and erase blind spots when covering news topics etc. Any ideas as to the feasibility?

>>1426
Thanks, I'm glad you like it!

1) The watcher hits up the 4chan API (in this form https://a.4cdn.org/m/thread/16086187.json), whereas the simple downloader hits the main html. My guess is that's what CF is blocking. If you had to set up headers or User-Agent to get the html to work, might want to replicate it for a.4cdn.org. Hydrus Companion may help here to copy cookies, not sure. If CF just blocks API hits from TOR, you may be out of luck.

2) Yeah, I bet the simple downloader looks for img tags or something, whereas webms are under 'video' or an iframe or something. Simple downloader can't do two things at once very well, I'm afraid! If you are feeling brave, you could try editing the simple parsing formula to try to get both img and (video), but I'm not sure if it is that clever.

3) Check out the 'page of pages'. Hit F9->special->page of pages. That'll let you nest your page tabs.

4) Thanks. Yeah, I'd like some basic support for more text formats, and 'any file' support, which will take some extra work for some internal technical reasons. But I'd like it. However, I do not think I can support nice text reading any time soon. I'm terrible at drawing lines, but I know I'll never have the time to make something anywhere near as good as Calibre, so the support will stay as an 'open externally' button, like PDFs now, for a long time.

5) Yeah, I absolutely want [+] expand/tree format for the tag list. That list is my own custom widget, so I have to code everything in it. But this is something I really want for myself too, and options for things like 'start with title tags collapsed'. And custom namespace sorting so you can put creator up top, that sort of stuff.

Having auto-scraping has been a long-time thought. I'm hesitant to make too many automatic searchers, since I like human eyeballs behind big decisions. I know some users, however, who have had big success writing their own scripts that fetch the current thread catalog, scan it for threads they want, and then pass them on to Hydrus using the Client API. If you are ok with scripting, you might like to play around with it: https://hydrusnetwork.github.io/hydrus/client_api.html

As always, I'm short on work time. When I eventually roll some of 4 + 5 out, let me know how it goes for you, and if you have any other problems or ideas.

>>1428
1) Will try that out, once I find the time. Thanks for the answer!

2) I'll experiment on that as well and will report back with how it goes.

3) Nice, awesome!

4) I'd be enough to merely have the works indexed inside Hydrus. You can always CTRL+E to open in your preferred reader/program with all the capabilities you need on the media itself. It's just so you can collect and organize the files, which Hydrus really blows anything else I know out of the water. There's some academic software that lets you organize your sources and stuff, but it's more of a list thing. Since you can add tons of metadata... well let's just say Hydrus and zlib would go hand in hand ;)

5) Yeah, collapsible tag trees would make stuff more easily readable. I know you can organize by custom namespaces and that is already really powerful, but it'd be cool to have tag:one tag:two tag:three below each other in the tag list and in the folder list as well, in "blocks" so to speak.

I feel you on automation. One misstep and you practically DDoS a site or flood your hard drive. I also didn't listen to your warning in the documentation haha. "and then THEY DO SO only to work through thousands of files", I felt that so hard and it brought a smile to my face. But once you have figured out what you want to parse daily etc. you then really want the automation for these specifica.

I really do appreciate your time man. Your project is one I always look into the documentation again and check the forums here. That's a rare exception for a really solid project. If you go the academic/library route, you might catch some licensing stuff to capitalize on your outstanding work, like for official use and stuff.