/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Posting mode: Reply

Check to confirm you're not a robot
Drawing x size canvas

Remember to follow the rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Manage Board | Moderate Thread

Return | Catalog | Bottom

Expand All Images

Version 372 Anonymous Board owner 10/16/2019 (Wed) 21:33:49 Id: 4fd51a [Preview] No. 221
https://youtube.com/watch?v=Klw-5eCji1w [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v372/Hydrus.Network.372.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v372/Hydrus.Network.372.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v372/Hydrus.Network.372.-.OS.X.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v372/Hydrus.Network.372.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v372.tar.gz

I had an ok week. I mostly fixed bugs and added little things to make for a 'clean' final wx release.

This is the last release for approximately four weeks.

all misc this week

You can now customise the height of the media viewer's animation scanbar, and the width of its nub, under options->media.

Right-clicking on a single tag in a taglist now lets you add/remove it from your current favourites. The whole menu there got a bit of cleanup.

I added a link to https://gitgud.io/koto/hydrus-archive-delete to the Client API help. It is a web browser based archive/delete filter that plugs into the Client API.

The new tag filter favourites system now has buttons to import/export tag filters from/to your clipboard. You can now share complicated filters with other clients or friends.

A couple of file import 'publishing' operations--when thumbnails from an import queue or subscription get added to a page--are now a little less laggy.

If global network traffic is paused, subscriptions will now not start work.

Anonymous Board owner 10/16/2019 (Wed) 21:34:41 Id: 4fd51a [Preview] No.222 del
full list

- petitions processing page:
- the selection taglist now displays the raw 'storage' tag view, before siblings are applied
- added a noneable spinctrl to control how many files are shown on a petition row double-click. it samples randomly and defaults to 256
- I think I fixed the issue where the petitions taglist sometimes hangs on to some old tags after a petition process event
- .
- the rest:
- you can now customise the animation scanbar height and nub width under options->media
- all users now see the number of open pages in the pages menu
- added approx total session 'weight' to the pages menu. this is an early test and will do more and update more frequently in future
- added add/remove tag to favourites to the taglist right-click menu
- collapsed the taglist right-click menu a little, as it was getting a bit tall
- added https://gitgud.io/koto/hydrus-archive-delete, a web browser archive/delete filter, to client api help
- added clipboard import/export buttons to the edit tag filter panel for the new favourites
- added 'open in a new page' to media viewer right-click menus, just to put the current single media in a new page
- fixed the url class edit panel not initialising with the new referral options correctly
- the call that publishes new subscription/import folder media to pages now does so more politely to the gui when the db is busy
- subscriptions will no longer start if global network traffice is paused
- the 'hard-replace siblings/parents' action under manage tags is now a local-only operation for tag repositories. clients with unusual sibling and parent application will no longer affect the repos they sync with
- on program shutdown, if a daemon takes more than thirty seconds to shutdown (which can happen in odd situations, like if a subscription run is paused by global network traffic pause, leading to shutdown deadlock), the client will stop waiting and continue with other shutdown tasks
- fixed an error with client 'already running' fast exit
- fixed a different error with server 'already running' exit choice
- updated some ffmpeg calls to fix certain OS problems
- fixed a help link to todolist recommendation and added link to new ptr guidelines

next week

There will not be a release for approximately four weeks. A user has been working for a long time on total conversion of hydrus's UI library, from wx to Qt, and we are now ready to pull the trigger. The program will still have the same layout, but elements such as buttons and scrollbars will have slightly different shadows and border widths and so on. The conversion to Qt will increase stability for all platforms, offer better support for newer technology like high-dpi displays, allow complete user-customisable themeing, and permit a complete MPV audio/video player embed straight into the media viewer.

I will take the code changes, crash-learn Qt's differences, check for and clean up some last bugs, and ensure I can make builds for Windows, Linux, and OS X. I will make status-update posts on Tuesdays and plan for 373 to be out on November 13th.

Anonymous 10/16/2019 (Wed) 23:07:32 Id: 64c60c [Preview] No.223 del
>allow complete user-customisable themeing
Thanks for your work hydrus dev, looking forward to Qt.

Anonymous 10/18/2019 (Fri) 23:30:10 Id: 207983 [Preview] No.225 del
is it possible to have a remove from view for files that the program can not open?

Im doing some culling based on tags where I hate almost everything that uses the tag, and i run into swf files, so I have to seperate them out every time.

so would it be possible to have a remove option that takes out unviewable files, and in the same vein, an 'unopenable files' search?

Anonymous Board owner 10/19/2019 (Sat) 17:29:49 Id: 4fd51a [Preview] No.229 del
Thanks, I will write this down. The new remove options are slightly hacked-in. My plan is to harmonise how the select and remove menus do their menu presentation and file-selection for actioning (the select menu does it cleverer atm), and then I can extend that one unified system to select files by mime.

For searching for unopenable files, would system:filetype work ok for that? It can be a pain checking all the boxes on that, but I'd like to do some work in the nearish future on having saved searches. It would be nice to have saved individual search predicates too, so you can quick-select things like 'has duration' or 'is a video with sound', or in your case, 'is one of x y z filetypes'.

Anonymous 10/22/2019 (Tue) 01:20:26 Id: 207983 [Preview] No.230 del
Its been a few days, so let me try to remember... I think the search option that just filters out anything that can't be opened would be great, this way I can look at the files that cant be opened and see why the hell they are in the archive in the first place, along with able to filter them out when I don't really want to deal with them, so a selectable option rather then check boxes would work for this.

when you save search results would it also be able to save a sort parameter to that as well? since the has audio came in, I sort my searches into 3 windows, 1 static, 1 audioless, and 1 with audio. saving the sort order in the search would just be a convenience, because after that I would use my normal sort of higher file size first,

Anonymous 10/22/2019 (Tue) 09:59:10 Id: 207983 [Preview] No.231 del
so, long story short, i'm an idiot and I put off deleting images, fell asleep and woke up to 'ran out of disc space'

so I have to ask, how does hydrus handle this?
did I lose the images or when space was made available did it copy?

Anonymous 10/23/2019 (Wed) 07:24:25 Id: 207983 [Preview] No.233 del
welp, just got this error

DatabaseError: database disk image is malformed
File "site-packages\wx\core.py", line 3259, in <lambda>
File "include\ClientThreading.py", line 501, in wx_code
File "include\HydrusThreading.py", line 769, in Work
File "include\HydrusData.py", line 1522, in call
self._func( *self._args, **self._kwargs )
File "include\ClientGUI.py", line 5385, in SaveLastSession
self._notebook.SaveGUISession( 'last session' )
File "include\ClientGUIPages.py", line 2917, in SaveGUISession
self._controller.WriteSynchronous( 'serialisable', session )
File "include\HydrusController.py", line 792, in WriteSynchronous
return self._Write( action, True, *args, **kwargs )
File "include\HydrusController.py", line 216, in _Write
result = self.db.Write( action, synchronous, *args, **kwargs )
File "include\HydrusDB.py", line 1028, in Write
if synchronous: return job.GetResult()
File "include\HydrusData.py", line 1705, in GetResult
raise e

Database Traceback (most recent call last):
File "include\HydrusDB.py", line 582, in _ProcessJob
result = self._Write( action, *args, **kwargs )
File "include\ClientDB.py", line 13740, in _Write
elif action == 'serialisable': self._SetJSONDump( *args, **kwargs )
File "include\ClientDB.py", line 11143, in _SetJSONDump
self._c.executemany( 'DELETE FROM json_dumps_named WHERE dump_type = ? AND dump_name = ? AND timestamp = ?;', [ ( dump_type, dump_name, timestamp ) for timestamp in deletee_timestamps ] )
sqlite3.DatabaseError: database disk image is malformed

with this message, wondering if this can be related to the 'i let disk space run out' problem I had earlier, or if this is something else.

Anonymous Board owner 10/23/2019 (Wed) 23:50:03 Id: 4fd51a [Preview] No.234 del
Saving sort with searches is a great idea, thank you. I will add that to that job.

For the 'search unviewable files', perhaps I can hack that into the system:filetype edit panel as a quick-select button.

I think hydrus will likely have not imported the images, treating the various imports as 'failed'. Once you are sorted again, I think you'll be able to retry them with the appropriate 'try again'/'retry failures' button for your downloader/subscription. I am assuming this is a downloader, in which case hydrus perhaps downloaded the image ok to your system disk temp space, but then failed to copy into your hydrus drive.

The solution is to free up space. If you are very tight, try moving some 'fxx' folders from install_dir/db/client_files (or if you have moved this, elsewhere), and then boot the client. It'll moan about the missing folders on boot, and give you a dialog to relocate them.

The 'malformed' error is much more worrying, and needs to be addressed before you can use the client again. Please check 'install_dir/db/help my db is broke.txt' for a full guide to figuring this out. I am also happy to assist you further here or one-on-one on discord/email if you like.

Since it happened so close, it sounds like the malformed error is related to the full disk. I do not think the full disk caused it, as SQLite extremely hardened against software failure--every instance of 'malformed' I have seen so far has been due to hardware failure--but perhaps the full disk triggered an emergency OS routine to free up temp sectors, or allocate less reliable sectors, or perform a weird TRIM or something that caused the drive to have a problem.

The .txt will guide you to figuring out which of the four db files are bad, and how to do a clone to try to recover them. With luck, the lack of disk space issues caused just the final db page to not be written right on the attempt to allocate new physical blocks. Again, please let me know if you need any special help, and I'd like to know how you get on here regardless.

Anonymous 10/24/2019 (Thu) 08:21:39 Id: 207983 [Preview] No.235 del
just sent this in email too, not sure which you will see first

Ok this is the only one that had something crop up, and at least from the looks of it, it doesn't seem serious.

going to go forward with clone client.db

for the time being going to also keep a new backup of everything prior to doing this too.

Anonymous 10/24/2019 (Thu) 19:00:13 Id: 207983 [Preview] No.236 del
well this is taking forever and looks like its getting little done, at
least relative to the size of client db

Anonymous 10/25/2019 (Fri) 05:31:26 Id: 207983 [Preview] No.237 del
by my estimate, this will take 60-70 days at this rate to do its thing, I don't think my computer has that kind of stability.

Anonymous 10/25/2019 (Fri) 23:33:15 Id: 207983 [Preview] No.239 del
Ok it finished, but I have a few concerns, namely the errors that it tossed.
An integrity check on the new db comes back clean, but the new
client.db is also about 1/22 the file size of the old one...

going to make an attempt to boot the new database soon, need to free
up some ram first

Anonymous 10/26/2019 (Sat) 01:16:24 Id: 207983 [Preview] No.240 del
Ok, tried booting it up with the client file, got this

tried booting it with the old client file, got the other message and
my session wouldn't load,
so I tried loading lass save session... wouldn't load
so I tried loading last exit session, that loaded.

I believe this is the error from the last save session load
DatabaseError: database disk image is malformed
Traceback (most recent call last):
File "include\ClientGUIPages.py", line 1545, in AppendGUISession
session = self._controller.Read( 'serialisable_named',
HydrusSerialisable.SERIALISABLE_TYPE_GUI_SESSION, name )
File "include\HydrusController.py", line 586, in Read
return self._Read( action, *args, **kwargs )
File "include\HydrusController.py", line 180, in _Read
result = self.db.Read( action, *args, **kwargs )
File "include\HydrusDB.py", line 1002, in Read
return job.GetResult()
File "include\HydrusData.py", line 1705, in GetResult
raise e
include.HydrusExceptions.DBException: DatabaseError: database disk
image is malformed
Database Traceback (most recent call last):
File "include\HydrusDB.py", line 578, in _ProcessJob
result = self._Read( action, *args, **kwargs )
File "include\ClientDB.py", line 10611, in _Read
elif action == 'serialisable_named': result =
self._GetJSONDumpNamed( *args, **kwargs )
File "include\ClientDB.py", line 6516, in _GetJSONDumpNamed
( version, dump, object_timestamp ) = self._c.execute( 'SELECT
version, dump, timestamp FROM json_dumps_named WHERE dump_type = ? AND
dump_name = ? ORDER BY timestamp DESC;', ( dump_type, dump_name )
sqlite3.DatabaseError: database disk image is malformed

Database Traceback (most recent call last):
File "include\HydrusDB.py", line 578, in _ProcessJob
result = self._Read( action, *args, **kwargs )
File "include\ClientDB.py", line 10611, in _Read
elif action == 'serialisable_named': result =
self._GetJSONDumpNamed( *args, **kwargs )
File "include\ClientDB.py", line 6516, in _GetJSONDumpNamed
( version, dump, object_timestamp ) = self._c.execute( 'SELECT
version, dump, timestamp FROM json_dumps_named WHERE dump_type = ? AND
dump_name = ? ORDER BY timestamp DESC;', ( dump_type, dump_name )
sqlite3.DatabaseError: database disk image is malformed

i'm out of ability to try things, it seems like all my ratings are
saved, so I am at most inconvenience on that end.

Anonymous Board owner 10/26/2019 (Sat) 20:52:55 Id: 4fd51a [Preview] No.242 del
Thank you, I replied to your email.

Anonymous 11/03/2019 (Sun) 18:39:19 Id: fede9d [Preview] No.258 del
A-are you still developing this software?
I just wanted to pop in to say that it is awesome!
Damn, I miss 8chan...

Anonymous 11/03/2019 (Sun) 22:03:10 Id: 207983 [Preview] No.259 del
software is still going, right now the ui is being transplanted into a different engine that should let it do more, one of the bigger things, it would allow the embedding of mpv and potentially bring audio on top of playback improvements, and if the dev disappears, im told there are people who would take over development. not sure how true that is, but it's a nice safety net incase something happens to hdev. on that note, the next release is coming next wednesday as a preview and the wednesday after for the next one for everyone, but I haven't been keeping up to much on what the current plan is.

Anonymous Board owner 11/04/2019 (Mon) 02:16:44 Id: 8185d1 [Preview] No.260 del
Yep, I am still here. >>259 is correct.

373 will be a special release that is taking more time. See the 'next week' section of the release post above. Once that is out, I'm back to normal weekly schedule.

I do short updates on my twitter here if you are interested in more frequent news: https://twitter.com/hydrusnetwork

It is mostly 'good day today chipping away at this big todo in front of me' at the moment. With luck and a couple of late nights, I should have a test release for advanced users out on the 6th, and a proper, bug-freer one (which if you are more new to the program, I recommend for you) out on the 13th.

8kun is trying to do their proper launch now. I expect the release on the 6th will be here, but if 8kun is stable and has a decent userbase again and /hydrus/ is migrated all correct by the 13th, I expect to move back.

Anonymous 11/05/2019 (Tue) 18:26:06 Id: f8be6f [Preview] No.261 del
Thanks for the clarifications guys, and you, Mr. Dev, are an awesome dev for actually taking time to respond to such questions, even if they were a bit stupid, sorry.
Have a great day!

Top | Return | Catalog | Post a reply