/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Boards | Catalog | Bottom

Check to confirm you're not a robot
Drawing x size canvas

Remember to follow the rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Version 364 Anonymous Board owner 08/15/2019 (Thu) 01:40:06 Id: 163926 [Preview] No. 28 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=oqRlJC-ckL4 [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v364/Hydrus.Network.364.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v364/Hydrus.Network.364.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v364/Hydrus.Network.364.-.OS.X.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v364/Hydrus.Network.364.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v364.tar.gz

I had a difficult week, but I got some nice work done. There is a new way to add complicated tag seaches, and repository processing is now a lot easier to work with.

complicated tag searching

Users in help->advanced mode will now see a new 'OR' button on their search pages' autocomplete dropdowns. Click this, and you will be able to paste a logically complicated pure-text query such as "( blue eyes and blonde hair ) or ( green eyes and red hair )", and it will automatically convert it to a tag query hydrus can understand! (It converts it to conjunctive normal form, or ANDs of ORs)

Message too long. Click here to view full text.

11 posts and 1 image omitted.

Anonymous Board owner 09/04/2019 (Wed) 02:52:16 Id: f6538c [Preview] No.46 del
Thank you for this feedback. I agree, and while I am happy with the initial version of siblings, they haven't fixed the problem of aliases completely. The current tags work I am doing will hopefully push this forward, but I think the real solution will be to develop a 'tag wiki' of some kind so tags have a bit of metadata of their own such as 'this is in japanese, it is not slang' so users can broadly choose what sort of style of tag they want to see as the sibling King that actually displays. It is not difficult to agree that 'vagina' and 'pussy' are synonyms, but it is impossible to agree which is best to display, so any future system will support more user choice over what is subjective. And since we are talking millions of different words here, there will have to be a rules-based approach.

I'll be rolling out good tag migration tomorrow and will start work on multiple local tag services next so you have multiple locations you have complete control of. I'd also like to add tag filters for a 'display' system so you can hide all 'filename:' or 'title:' tags or whatever. I want it to be easy to pull just the tags you want from a big source like the PTR and hide the stuff you don't like.

Anonymous Board owner 09/04/2019 (Wed) 02:55:36 Id: f6538c [Preview] No.47 del
Once I am done with this tags work. I haven't pinned down the limits of what I want to do yet, so I can't give a very certain timeframe, but we should be back to normal schedule once all the fires are out. My roughest guess is another 10 weeks or so.

Anonymous 09/04/2019 (Wed) 06:05:38 Id: ba7794 [Preview] No.50 del
>so if your temp dir's partition has less than 5GB space, that could be it.
That was the case!
My tmpfs folder was 4.9GB so I made hydrus's tmp folder somewhere else.
Then everything works!

Anonymous 09/04/2019 (Wed) 18:55:41 [Preview] No.51 del
can i use hydrus to access uncensored gelbooru? I tried adding the API Access Credentials url string to gallery url generator and to url classes.

Anonymous 09/05/2019 (Thu) 02:56:23 Id: 73eb4d [Preview] No.57 del
Afaik it needs some special cookie that is set with a form and is independent of account, so the login alone doesn't do it. Some users are working on an updated login script I think, but I don't know much about it.

For now, if you check your web browser's gelb cookies and copy across the missing cookie to hydrus under network->data->review session cookies I think that'll do it. The cookie has a simple english name something like 'letmein' and a value of 'yes' or 'true' or something I think.

Anonymous 08/29/2019 (Thu) 08:14:28 Id: d149fe [Preview] No. 41 [Reply] [Last 50 Posts]
>> new board, how do
>> find suggestions etc
>> cry a lot
>> duration for vids a thing maybe?
>> nope?
>> could it be ... push toward vid/audio support maybe, sometime when?
>> plead with the hydrus god's

Anonymous Board owner 09/04/2019 (Wed) 03:06:04 Id: 887cdf [Preview] No.49 del
I apologise for the inconvenience. I hope the imageboard instability lifts soon and we can settle back to normal.

Can you say more what you mean by 'duration for vids'?

Release Tomorrow! Anonymous Board owner 09/04/2019 (Wed) 03:01:19 Id: 86b2a1 [Preview] No. 48 [Reply] [Last 50 Posts]
I had a great week finishing off what I started last week. There is now a comprehensive way to move and delete large numbers of tags between services and external archive files. I also fixed a variety of bugs, reworked the main right-click menu on media (including extensions for the 'remove' command, such as 'remove all archived'), and added a couple of shortcuts to perform mass-page refreshes.

The release should be as normal tomorrow.

Anonymous 09/03/2019 (Tue) 06:24:53 Id: c5f4ef [Preview] No. 43 [Reply] [Last 50 Posts]
Hello dev
Is there filesize limit in hydrus?
Hydrus yields an error when I'm going to import a file bigger than 5GB
Of cource I have enough space on HDD

Here's error message and I'm using Hydrus ver 365 and Arch linux
filesystem is ext4

[Errno 28] No space left on device
Traceback (most recent call last):
File "/home/erdos/ssd/hydrus/include/HydrusPaths.py", line 719, in MirrorFile
shutil.copy2( source, dest )
File "/usr/lib/python3.7/shutil.py", line 266, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.7/shutil.py", line 122, in copyfile
copyfileobj(fsrc, fdst)

Message too long. Click here to view full text.

(92.06 KB 450x409 01-success-kid-c.jpg)
Anonymous Global volunteer 08/17/2019 (Sat) 12:24:50 Id: 0473cb [Preview] No. 40 [Reply] [Last 50 Posts]
I'm posting this so you guys won't create conspiracy theories. I know, my hope us futile.
If everything goes well soon the site's and our problems will be resolved. But they will turn the site read-only mode for a while. I have no information how long the process will take. Keep calm and be patient.


Release Tomorrow! Anonymous Board owner 08/14/2019 (Wed) 06:05:30 Id: 1fbcab [Preview] No. 26 [Reply] [Last 50 Posts]
I had a difficult week, but I got some good work done. As well as some fixes and improvements to the new 'has audio' detector and a bunch of other bug fixes, there is a neat new way to input logically complicated tag searches, and I reshaped the repository processing pipeline to no longer block the program--you can now keep browsing your files while repository work is going on!

The release should be as normal tomorrow. I have some messages to catch up on, which I will try to clear out tomorrow.

Database Encrypted Sync Anonymous 08/07/2019 (Wed) 18:59:51 Id: 01e229 [Preview] No. 3 [Reply] [Last 50 Posts]
I haven't seen this discussed so I thought I'd make a thread on it.

We are all aware that Hydrus runs fine inside a Truecrypt container, but having one huge file can be a problem.

For instance if you backup your 500GB database inside a truecrypt container and then download even just 1 image you basically need to backup the whole 500GB all over again if you run an automated backup to an external HDD or a NAS. Another problem with Truecrypt/Veracrypt volumes is you have to set the container size at the time of making it, leaving it too big or too small all too easily.

Well I've been experimenting with different crypto programs that feature encrypting directories while keeping files separate. The filenames and contents are unreadable, but they are still their own files so syncing and backup programs know exactly what to copy and what is the same.

The problem is that Hydrus doesn't want to run at all within these encrypted directories, but luckily we can have our installed client separate to the db folder.
>install an encryption program (I used cppcryptfs but there are a bunch for various platforms)
>create a new encrypted folder on any drive
>mount it as a Drive letter (you will need to use the same Drive letter each time)
>Move your Hydrus db folder to that mounted drive (or you can make a new one by ignoring this step)
>create a shortcut to your client.exe adding -d="path to db folder on virtual drive" (e.g "C:\Hydrus Network\client.exe" -d="Z:\db")
>use shortcut

it should find your database and start like normal, with any new files being encrypted as their own files that can be synced individually.

Message too long. Click here to view full text.

Anonymous Board owner 08/09/2019 (Fri) 15:57:20 Id: 8e81bb [Preview] No.20 del
Hey, I can't speak cleverly about encryption systems that keep all files separate as I don't have experience with them, but if you would like to use traditional containers, a good way to backup is to backup the mounted contents to another mount. So:

Container 1, 500GB: Has hydrus in it
Container 2, 500GB: Has hydrus backup in it

Mount 1
Mount 2
Set up a program like FreeFileSync to do a 'mirror' run between mounted file structures of 1 to 2
Wait five minutes for that to find differences
Run the sync, so 2 is now a mirror of 1, usually takes a couple of minutes
Dismount 2
2 is now a mirror of 1 without having to replace the whole file.

Do that once a week or so, and 2 will stay a safe backup of 1 without much work. I think you'd be able to automate it somewhat, if you are into that.

Version 363 Anonymous 08/08/2019 (Thu) 00:04:35 Id: ff6806 [Preview] No. 5 [Reply] [Last 50 Posts]
https://youtube.com/watch?v=VWY8rYPuftY [Embed]
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v363/Hydrus.Network.363.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v363/Hydrus.Network.363.-.Windows.-.Installer.exe
os x
app: https://github.com/hydrusnetwork/hydrus/releases/download/v363/Hydrus.Network.363.-.OS.X.-.App.dmg
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v363/Hydrus.Network.363.-.Linux.-.Executable.tar.gz
tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v363.tar.gz

I had a good week of work. There is new 'has audio' detection for videos, extensions to the client api, and plenty of bug fixes.

has audio

The client can now detect if a video has an audio channel and, if so, presents that information with the unicode character 🔊 on its thumbnail-right click and in the media viewer! Audio files and swf files will get this as well. You can now also search for this status (or exclude it) with the new 'system:has audio' search predicate.

Message too long. Click here to view full text.

7 posts omitted.

Anonymous 08/08/2019 (Thu) 21:03:09 Id: aff939 [Preview] No.15 del
hey dev,
I imported some webms, then deleted them from my inbox because I forgot to add their filenames as tags. But when I went to import them a second time, they don't get imported? It says they've been previously deleted? Which is technically true, i guess, but how would I revert that? I just wanted to re-import them, but with tags this time.

Anonymous Board owner 08/09/2019 (Fri) 15:32:15 Id: 466200 [Preview] No.16 del
There's a bug that files that get a metadata reparse sometimes(?) do not want to receive rating/tag/archive updates. The metadata object is being uncoupled somehow at the thumb level on the update event. I am not sure how long this has been in the client, but now file metadata reparsing is ramping up, I'll make sure to sort this out for 364.

Thanks. Yeah, I vacillated on whether or not to default 'has audio' on or off, and whether to have a 'not yet ready' state. To keep things simple, I decided to have everything appear as it did, and for new/correct info to slowly come in over time. I will think about building in better 'uncertain' state the next time I do something like this.

I agree about the slow and inconvenient retroactive parsing. I think I'd like the new file maintenance system to maybe have two processing states--one where it can do some heavy work during idle time, but a little more frequently than at present, and one where it is always working, but silently, something like 1 file every 15 seconds or something. 0.2s processing time to generate a new thumb or do some metadata work isn't inconvenient when it is spread out that much, but I'd have options so you can tune it to whatever works for your situation.

Thank you! I have noticed a couple myself as well. I got my 'is silent' detector to work for some files, but now not for these others, so I appreciate the examples here. I'll tweak my code and try to deal with these ones--perhaps some metadata is sneaking through, or some odd EOF characters or something.

Anonymous Board owner 08/09/2019 (Fri) 15:41:38 Id: 466200 [Preview] No.17 del
Ah, actually, that e621 file is a swf. For now, I am just defaulting all flesh to 'has audio', because trying to get accurate metadata out of that format is difficult. When I add audio playback, I'm going to go to a paradigm of a global silence/noisy mode, and while flash can't be embedded any more in the new wx, I want to default potential-but-unknown noisy types away from the known safely silent side.

I'll queue up a job to see if ffmpeg or anything else I have access to can pull any audio files or anything from flash. Maybe it can see some embedded mp3s or something, and I can notice the lack of them. But even then, it is a painful format--the thumbnail generation is black magic, and that fails quite frequently.

Anonymous Board owner 08/09/2019 (Fri) 15:46:26 Id: 466200 [Preview] No.18 del
Hey, there isn't an excellent way to do this yet, but it is totally possible. I have a job queued up to write some proper help on this, since the same question comes up, but for now:

To do files, just do a mass export and import. You can either shut the client down and copy/import all the 'fxx' folders in your client_files folder to your new machine/client, or you can set up a big export with a big search and thumbnail right-click->share->export->files, or you can set up a one-time export folder for 'system:everything' under file->import and export folders.

To do tags, turn on help->advanced mode and hit 'advanced service-wide update' button under services->review services. This lets you import/export all tags on your 'local tags' to a portable Hydrus Tag Archive file.

Let me know if you run into any trouble! These are big operations, so it is worth making a backup beforehand, just in case something goes wrong.

Anonymous Board owner 08/09/2019 (Fri) 15:50:33 Id: 466200 [Preview] No.19 del
Hey, by default, the client excludes files it has previously seen imported and deleted. This functionality is controlled through any import context's file import options. Up top on the file import options's edit panel is 'exclude previously deleted files'--uncheck that and your import should go through ok.

I generally recommend that people keep this on and only turn it off for one-time jobs like this, but if you prefer it off all the time, you can set the default file import options for new import contexts under file->options->importing.

By the way, if this happens again, you don't have to delete the files from the client if they came in wrong the first time--you can just run the import again, and hydrus will notice the duplicates already in there and add the new tags to them (and work a bit faster, too, since it doesn't have to do the file import itself).

Anonymous Board owner 08/07/2019 (Wed) 02:23:57 Id: 8bdd04 [Preview] No. 1 [Reply] [Last 50 Posts]
never give up