- the 'set_notes' command now checks the type of the notes Object. it obviously has to be string-to-string
- the 'get_thumbnail' command should now never 404. if you ask for a pdf thumb, it gives the pdf default thumb, and if there is no thumb for whatever reason, you get the hydrus fallback thumbnail. just like in the client itself
- updated client api help to talk about these
- updated the unit tests to handle them too
- did a pass over the client api help to unify indent style and fix other small formatting issues
- client api version is now 28

next week

I am feeling good about multiple local file services. Most of the cleanup this week was for that, and now there are only about three things left to do before we can start playing with it for real--UI and some importer code to handle imports to multiple locations, UI to present deletes and undeletes for multiple locations, and UI and db code to do move/copy across locations. I'll push on these in the coming weeks.

Next week will be a 'small jobs' week, and I would like to catch up on github issues in particular.

I had an ok week with a variety of work. I fixed some bugs, tweaked some UI (including a neat change to shift+select of thumbnails), improved the new file history chart, and added to the Client API.

The release should be as normal tomorrow.

full list

- misc:
- the network engine now parses the 'last-modified' response header for raw files. if this time is earlier than any parsed source time, it is used as the source time and saved to the new 'domain modified time' system. this provides decent post time parsing for a bunch of sites by default, which will also help for subscription timing and similar
- to get better apng duration, updated the apng parser to count up every frame duration separately. previously, if ffmpeg couldn't figure it out, I was just defaulting to 24 fps and estimating. now it is calculated properly, and for variable framerate apngs too. all apngs are scheduled for a metadata regen this week. thanks to the user who submitted some long apngs where this problem was apparent
- fixed a bug in the network engine filter that figures out url class precedence. url classes with more parameters were being accidentally sorted above those with more path components, which was messing with some url class matching and automatic parser linking
- improved the message when an url class fails to match because the given url has too few path components
- fixed a time delta display bug where it could say '2 years, 12 months' and similar, which was due to a rounding issue on 30 day months and the, for example, 362nd day of the year
- fixed a little bug where if you forced an archive action on an already archived file, that file would appear to get a fake newer archived timestamp in UI until you restarted
- updated the default nitter parsers to pull a creator tag. this seemed to not have been actually done when previously thought
- the image renderer now handles certain broken files better, including files truncated to 0 size by disk problem. a proper error popup is made, and file integrity and rescan jobs are scheduled
- .
- file history chart:
- for a long time, a user has been generating some cool charts on file history (how many files you've had in your db over time, how many were deleted, etc...) in matplotlib. you may have run his script before on your own database. we've been talking a while about integrating it into the client, and this week I finally got around to it and implemented it in QtCharts. please check out the new 'view file history' underneath Mr Bones's entry in the help menu. I would like to do more in this area, and now I have learned a little more about QtCharts I'd like to revisit and polish up my old bandwidth charts and think more about drawing some normal curves and so on of other interesting data. let me know what you think!
- I did brush up a couple things with the bandwidth bar chart already, improving date display and the y axis label format
- .
- client api:

Here's my chart! Two notes:

1) My inbox is a little higher than it should be because I rescheduled everything for a second scan some years ago.
2) Guess when 'many queries on one subscription' was added. :^)

I had a good week. I cleaned a heap of code, fixed some bugs, brushed up the new file history graph, and wrote some small extensions to the Client API.

The release should be as normal tomorrow.

full list

- domain modified times
- the downloader now saves the 'source time' (or, if none was parsed, 'creation time') for each file import object to the database when a file import is completed. separate timestamps are tracked for every domain you download from, and a file's number can update to an earlier time if a new one comes in for that domain
- I overhauled how hydrus stores timestamps in each media object and added these domain timestamps to it. now, when you see 'modified time', it is the minimum of the file modified time and all recorded domain modified times. this aggregated modfified time works for sort in UI and when sorting before applying system:limit, and it also works for system:modified time search. the search may be slow in some situations--let me know
- I also added the very recent 'archived' timestamps into this new object and added sort for archived time too. 'archived 3 minutes ago' style text will appear in thumbnail right-click menus and the media viewer top status text
- in future, I will add search for archive time; more display, search, and sort for modified time (for specific domains); and also figure out a dialog so you can manually edit these timestamps in case of problems
- I also expect to write an optional 'fill in dummy data' routine for the archived timestamps for files archived before I started tracking these timestamps. something like 'for all archived files, put in an archive time 20% between import time and now', but maybe there is a better way of doing it, let me know if you have any ideas. we'll only get one shot at this, so maybe we can do a better estimate with closer analysis
- in the longer future, I expect import/export support for this data and maintenance routines to retroactively populate the domain data based on hitting up known urls again, so all us long-time users can backfill in nicer post times for all our downloaded files
- .
- searching tags on client api:
- a user has helped me out by writing autocomplete tag search for the client api, under /add_tags/search_tags. I normally do not accept pull requests like this, but the guy did a great job and I have not been able to fit this in myself despite wanting it a lot
- I added some bells and whistles--py 3.8 support, tag sorting, filtering results according to any api permissions, and some unit tests
- at the moment, it searches the 'storage' domain that you see in a manage tags dialog, i.e. without siblings collapsed. I can and will expand it to support more options in future. please give it a go and let me know what you think
- client api version is now 26
- .
- misc

- if you set some files as 'alternates' when they are already 'duplicates', this now works (previously it did nothing). the non-kings of the group will be extracted from the duplicate group and applied as new alts
- added a 'BUGFIX' checkbox to 'gui pages' options page that forces a 'hide page' signal to the current page when creating a new page. we'll see if this patches a weird error or if more work is needed
- added some protections against viewing files when the image/video file has (incorrectly) 0 width or height
- added support for viewing non-image/video files in the duplicate filter. there are advanced ways to get unusual files in here, and until now a pdf or something would throw an error about having 0 width

next week

Back to multiple local file services, which is in endgame. I have a ton of ancient file handling code to simply clean to newer standards.

I had a good week. As well as some general fixes and more file modified time work, there is also a user-written expansion to the Client API that adds 'notes' editing support and a bit of fun.

The release should be as normal tomorrow.

- I added a list of some quick links back in to the main index page of the help
- I wrote an unlinked 'after_distaster' page for the help that collects my 'ok we finished recovering your broken database, now use your pain to maintain a backup in future' spiel, which I will point people to in future
- .
- misc:
- fixed a bug where changes to the search space in a duplicate filter page were not sticking after the first time they were changed. this was related to a recent 'does page have changes?' optimisation--it was giving a false negative for this page type (issue #1079)
- fixed a bug when searching for both 'media' and 'preview' view count/viewtime simultaneously (issue #1089, issue #1090)
- added support for audio-only mp4 files. these would previously generally fail, sometimes be read as m4a. all m4as are scheduled for a metadata regen scan
- improved some mpeg-4 container parsing to better differentiate these types
- now we have great apng detection, all pngs with apparent 'bitrate' over 0.85 bits/pixel will be scheduled for an 'is this actually an apng?' scan. this 0.85 isn't a perfect number and won't find extremely well-compressed pixel apngs, but it covers a good amount without causing a metadata regen for every png we own
- system:hash now supports 'is' and 'is not', if you want to, say, exclude a list of hashes from a search
- fixed some 'is not' parsing in the system predicate parser
- when you drag and drop a thumbnail to export it from the program, the preview media viewer now pauses that file (just as the full media viewer does) rather than clears it
- when you change the page away while previewing media with duration, the client now remembers if you were paused or playing and restores that state when you return to that page
- folded in a new and improved Deviant Art page parser written by a user. it should be better about getting the highest quality image in unusual situations
- running a search with a large file pool and multiple negated tags, negated namespaces, and/or negated wildcards should be significantly faster. an optimisation that was previously repeated for each negated tag search is now performed for all of them as a group with a little inter-job overhead added. should make '(big) system:inbox -character x, -character y, -character z' like lightning compared to before
- added a 'unless namespace is a number' to 'tag presentation' options, which will show the full tag for tags like '16:9' when you have 'show namespaces' unticked
- altered a path normalisation check when you add a file or thumbnail location in 'migrate database'--if it fails to normalise symlinks, it now just gives a warning and lets you continue. fingers crossed, this permits rclone mounts for file storage (issue #1084)

- some cleanup:
- replaced last instances of EVT_CLOSE wx wrapper with proper Qt code
- did a heap of very minor code cleanup jobs all across the program, mostly just to get into pycharm
- clarified the help text in _options->external programs_ regarding %path% variable
- .
- pycharm:
- as a side note, I finally moved from my jank old WingIDE IDE to PyCharm in this release. I am overall happy with it--it is clearly very powerful and customisable--but adjusting after about ten or twelve years of Wing was a bit awkward. I am very much a person of habit, and it will take me a little while to get fully used to the new shortcuts and UI and so on, but PyCharm does everything that is critical for me, supports many modern coding concepts, and will work well as we move to python 3.9 and beyond

next week

The past few months have been messy in scheduling as I have dealt with some IRL things. That's thankfully mostly done now, so I am now returning to my old schedule of cleanup/small/medium/small week rotation.

Next week will be a 'medium size' job week. I'm going to lay the groundwork for 'post time' parsing in the downloader and folding that cleverly into 'modified date' for searching and sorting purposes. I am not sure I can 'finish' it, but we'll see.

I had a good week bringing two neat new features: a user has implemented tag autocomplete search for the Client API, and I managed to get 'post time' saving from the downloader system to the database and augmenting 'modified time'.

The release should be as normal tomorrow.

Anyone here know what's wrong with my Hydrus Companion?
Sending tabs from Gelbooru it says it sends them successfully, but Hydrus says 400 Unknown Error.

Hey, I finally moved to new programming software this week (WingIDE to PyCharm). It was a little jarring dealing with new UI and shortcuts and a billion new settings, so I mostly stuck to simple code cleanup to get to grips with it. I don't have much exciting in my changelog beyond a tweak to system:hash, so rather than put out a thin build tomorrow, I will do some more work instead and move the release on a week.

If you haven't seen it, though, please check out the new help a user put together: https://hydrusnetwork.github.io/hydrus/

It has nice features like search and tables of contents and will be easier to edit in future. 475 will have it too for the local copy and should be out on the 2nd of March.

>>1235
If you check your 'client - date.log' file in the install_dir/db folder, and go to the bottom and look for the timestamp when this happened (and maybe ctrl+f for "400" or "unknown error", whatever text you have), can you see any 'traceback' information for the errors here? Can you clip and pastebin them or just post here? That will show more information on what is actually failing here.

If you are getting popups in the client UI, it should let you just copy the traceback there, too.

>>1236
I go to the hydrus page by typing hy and then enter. going there it's nicer than it was before. however it took quite a while for me to find the links to the various places hydrus is, would it be possible to copy at least the downloads and links to the places you make update posts to the homepage?

>>1237
Thank you, yes, I will. When we collapsed it down a level, the old index with the links go removed. I'll add something back in.

I had a good couple of weeks. I updated my behind the scenes environment and cleared out a wide variety of misc work--bug fixes and little improvements. The built release will also get a local copy of the nice new help a user put together.

The release should be as normal tomorrow.

>>1227
meant for >>1221

>>1227
Great, thanks for letting me know. For sound on your video, it may be that you have mpv now but because you once did not, hydrus reverted you back to the old renderer in the options.

Please hit help->about and see if you have mpv (it will give you a popup error if you don't). If you do, then hit up file->options->media, and for video and animation, change the 'media show action', if it is set to 'show with native hydrus viewer' to 'show using mpv'.

If you don't have mpv in help->about, let me know.

>>1229
The about says "mpv: : is not currently available on macOS" and I didn't have the option to switch to mpv in the media options. I then brew installed mpv and it gives the same message in the about, but now it lets me choose mpv in the media options. However, when I choose it and try to play a video file the program hangs. Also thanks for the help with updating, I was worried my db was boned

>>1230
Damn, I had forgotten that was true of the App build. I know some macOS guys who have mpv working, but they are running from source.

I'm changing tomorrow's build so it doesn't say that anymore if mpv is actually detected and loaded. Can you hit help->about in 474 sometime and let me know what 'api version' it says you have? Getting the correct versions so python side and mpv side can talk happily to each other has been tricky on Linux, I'll bet it is the same for macOS. The Windows build currently uses mpv 1.109.

I had a great week working on a whole bunch of little fixes and improvements.

The release should be as normal tomorrow.

>>1219
I've narrowed down the error to the update from 438 to 439

>>1218
Thank you, I am glad you like it!

>>1219
>>1220
Thank you for this report. I am sorry for the trouble. I know that around that time (441), we moved to builds on Github. This was because my old macOS laptop (I think it was 10.13 High Sierra) was so old the things it made were incompatible with the new Big Sur. We now build on a Big Sur machine on the cloud. 439 isn't exactly when that happened, but I know I started work on newer version requirements for libraries around that time, so I am afraid I think this might just be a version incompatibility problem. Searching "LSOpenURLsWithRole 10810" shows a variety of similar versioning problems.

However there are also some things like this: https://www.mathworks.com/matlabcentral/answers/460920-why-do-i-get-the-error-lsopenurlswithrole-failed-with-error-10810-when-trying-to-install-matlab

Maybe this is related to security? Is your macOS set up with default security rules. I know you normally have to enable 'Allow apps downloaded from anywhere' as the Readme in the App dmg says.

I am not a macOS expert however. I thought that Mojave (10.14?) was compatible with Big-Sur-built Apps. I may be wrong, or it may just be more complicated than I understand. Do you happen to have a special version of python installed on your machine? Some of the searching I did on that error suggested a conflicting python install might cause it somehow.

I assume if you just run the App by double-clicking its icon (which would normally make a database in ~/Library I think), that doesn't work either?

If it seems like this is just a versioning problem--that Mojave (or your Mojave) is too old to run the Big Sur build, I think your best shot is to see about updating your machine to a later macOS. When we move to Qt 6 over this year, I am afraid I will have to say the same about Windows 7. Unfortunately, macOS seems to retire older OSes faster than Windows. Another potential answer here, if you have some python experience, although it sucks, is to run from source. It may be possible to keep using an older version of Qt, I imagine around 5.12 (check your help->about in v438 to see what Qt seems ok for you), if you are willing to go through the work of setting up a virtual environment and testing out different library versions. Here is my page on that, if you are interested: https://hydrusnetwork.github.io/hydrus/help/running_from_source.html

I had a mixed week--unfortunately, some IRL stuff reduced my work time. I did however clean some code, fix a couple bugs, and integrate a cool new quick-navigation widget that a user wrote.

The release should be as normal tomorrow.

How do I give it the cookie?
Why does it need the cookie to log in when I gave it the login info already?
v464, if that matters

>>1223
Hmm, this error basically means that as it tried to log in, the site did not give a cookie that the login script expected to be given. This suggests the site has changed its login process since that script was written, or maybe an unusual error like your login password was wrong or something or the login page now has a captcha, so the login just failed.

For all complicated login situations, I recommend Hydrus Companion now. It lets you copy your cookies right from your logged-in browser to Hydrus using the Client API:

https://gitgud.io/prkc/hydrus-companion

full list

- times:
- if you have file viewing stats turned on (by default it is), the client will now track the 'last viewed time' of your files, both in preview and media viewers. a record is only made assuming they pass the viewtime checks under _options->file viewing statistics_ (so if you scroll through really quick but have it set to only record after five seconds of viewing, it will not save that as the last viewed time). this last viewed time is shown on the right-click menu with the normal file viewing statistics
- sorting by 'import time' and 'modified time' are moved to a new 'time' subgroup in the sort button menu
- also added to 'time' is 'last viewed time'. note that this has not been tracked until now, so you will have to look at a bunch of things for a few seconds each to get some data to sort with
- to go with 'x time' pattern, 'time imported' is renamed to 'import time' across the program. both should work for system predicate parsing
- system:'import time' and 'modified time' are now bundled into a new 'system:time' stub in the system predicates list. the window launched from here is an experimental new paged panel. I am not sure I really like it, but let's see how it works IRL
- 'system:last view time' is added to search the new field! give it a go once you have some data
- also note that the search and sort of last viewed time works on the 'media viewer' number. those users who use preview or combined numbers for stuff, let me know if and how you would like that to work here--sort/search for both media and preview, try to combine based on the logic in the options, or something else?
- .
- loading serialised pngs:
- the client can now load serialised downloader-pngs if they are a perfect RGB conversion of an original greyscale export.
- the pngs don't technically have to be pngs anymore! if you drag and drop an image from firefox, the temporary bitmap exported and attached to the DnD _should_ work!
- the lain easy downloader import now has a clipboard paste button. it can take regular json text, and now, bitmap data!
- the 'import->from clipboard' button action in many multiple column lists across the program (e.g. manage parsers) (but not every list, a couple are working on older code) also now accepts bitmap data on the clipboard
- the various load errors here are also improved

- misc:
- mouse release events no longer trigger a command in the shortcuts system if the release happens more than about 20 pixels from the original mouse down. this is tricky, so if you are into clever shortcuts, let me know how it works for you
- the file maintenance manager (which has been getting a lot of work recently with icc profiles, pixel dupes, some thumb regen, and new audio channel checks), now saves its work and publishes updates faster to the UI, at least once every ten seconds
- the sort entries in the page sort control are now always sorted according to their full (type, name) string, and the mouse-wheel-to-navigate is now fixed to always mirror this
- improved some 'delete file reason' handling. currently, a file deletion reason should only be applied when a file is entering trash. there was a bug that force-physical-deleting files from trash would overwrite their original deletion reason. this is now fixed. the advanced delete files dialog now disables the whole reason panel better when needed, never sends a file reason down to the database when there should be no reason, disables the panel if all the files are in the trash, and at the database level when file deletion reasons are being set, all files are filtered for status beforehand to ensure none are accidentally set by other means. I am about to make trash more intelligent as part of multiple local file services, so I expect to revisit this soon
- the new ICC Profile conversion no longer occurs on I or F mode files. there are weird 32/64 bit monochrome files, and mode/ICC conversion goes whack with current PIL code
- replaced the critical hamming test in the duplicate files system with a different bit-counting strategy that works about 9% faster. hamming test is used in all duplicate file searching, so this should help out a tiny bit in a lot of places
- .
- boring cleanup:
- cleaned up how media viewer canvas type is stored and tested in many places
- all across the program, file viewing statistics are now tracked by type rather than a hardcoded double of preview & media viewer. it will take a moment to update the database to reflect this this week
- cleaned up a ton of file viewing stats code
- cleared out the last twenty or so uses of the old 'execute many select' database access routine in favour of the new lower-overhead and more query-optimisable temporary integer tables method

next week

Back to multiple local file services. A widget to select multiple services in file search, and I'll start on a trash that can handle deletes from and undeletes back to multiple locations.

I had a good week. I fixed a heap of bugs, added a couple new ways to scale thumbnails, and finished the basics of multiple domain file search.

The release should be as normal tomorrow.