/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Posting mode: Reply

Check to confirm you're not a robot
Name
Email
Subject
Comment
Password
Drawing x size canvas
File(s)

Remember to follow the rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Manage Board | Moderate Thread

Return | Catalog | Bottom

Expand All Images


Version 446 Anonymous Board owner 07/07/2021 (Wed) 21:26:32 Id: 76b34f [Preview] No. 1099
https://youtube.com/watch?v=asojeparbK0 [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v446/Hydrus.Network.446.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v446/Hydrus.Network.446.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v446/Hydrus.Network.446.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v446/Hydrus.Network.446.-.Linux.-.Executable.tar.gz

I had an ok week. The client does not have a huge changelog this week, but the server has a neat privacy improvement.

The PTR is going to be doing a heap of maintenance this week. It will be 'busy' a lot, approximately one hour of busy and then three hours free. Please bear with it, and if you have a million mappings to upload, I recommend you just give it a break and come back later. I am not totally sure how long it will take. Best case it is a day, worst case it might take four or five.

null account

tl;dr: The hydrus server is now even more anon. You don't have to do anything.

As the PTR has moved to multiple accounts, we've had several good discussions about privacy. Separate accounts, despite being anon, could potentially leave a fingerprint of preferences on the server. If the server were to fall into bad hands many years from now, someone could mine those records--maybe mixed with one time you casually said 'yeah, I added that sibling'--and perhaps derive something from it.

There is no technical need to remember which account uploaded what long term, so now all hydrus servers completely anonymise their content after a certain duration, default 90 days. A new non-useable 'null' account takes possession of files, tag mappings, siblings, or parents after the delay, letting the original uploaders be forgotten. Janitors will still have time to work on recent account-based problems, but the historical record works just like the old shared public account: all merged together.

I have updated the privacy document in the help to talk a little about it. As long as you do not tag your own face in pictures or something, I think we are pretty great now, especially if you use a VPN.

https://hydrusnetwork.github.io/hydrus/help/privacy.html#account_history

It will take some time to anonymise the PTR or any other big server, as it has to go through its whole historical record to catch up. Please bear with it.


Anonymous Board owner 07/07/2021 (Wed) 21:26:54 Id: 76b34f [Preview] No.1100 del
full list

- misc:
- fixed a typo bug in the latest pending upload routine when it was cancelled/errors out early
- fixed a problem with the new subscription gap downloader, where when the page opens with the first query, it could sometimes assign 'already in db' to items in that query that were actually 'successful'. some other downloaders may have been rarely hit by this, but it was mostly the gap downloader
- the client _should_ now support a service host that has path components (e.g. one hosted on a proxy), like myserver.com/hydrus_repo. the port will now be correctly inserted in the address before all requests. hydrus and ipfs both should work, fingers crossed
- when an admin modifies the account types, the server now only prints the 'updated account type' log record if there were actual changes
- .
- downloader UI:
- the confusingly named 'file import status' and 'gallery search log' in the downloader system are now renamed 'file log'/'search log'/'check log' for the file import queue, gallery downloader, and watcher respectively. the 'table' bitmap buttons are also replaced with simple easy (and easy to refer to!) label buttons.
- when you open the file/search/check logs from the downloader page list right-click menu, they now spawn properly inside regular windows, not modal dialogs (which were inhibiting interaction with the rest of the program while open)
- relabelled the awkward 'even if url/hash recognised' checkboxes in file import options. 'do not skip if' becomes 'force download even if', and the text matches that in tag import options for page content. also improved the tooltip on these checkboxes
- all of the downloader layout boxes have also been renamed and harmonised with each other. gone are overly technical 'import queue' and 'gallery parsing'. now it is generally 'imports' up top and 'search' or 'checker' etc... below
- layouts have also been harmonised a little. the url downloader page now has boxes for file vs search URLs, the hard drive import pause button is moved up as other pages have, and several off-by-a-pixel sizer layouts have been fixed
- .
- null account:
- to further improve server privacy, particularly after the PTR's multiple account switch, all repositories now forget which accounts uploaded which content after a certain age. by default it is 90 days, but you can check in _review services_ once a server updates. this defends against a variety of hypothetical attacks where someone very clever gains access to the raw server database files, maybe years from now, and tries to crawl its anonymous account history for derivable information--now there is no history!
- it will take some time to retroactively scrub a huge server like the PTR. for the PTR, it is mostly a relative no-op of moving account ids from the old public shared account to a new 'null' account, but it'll still be about 1.2 billion rows! this happens in the background, so the server will still be useable most of the time, but it will have spikes of 'busy' for about one hour every four (i.e. one hour of mostly busy, three hours of free), probably for several days. it may be a pain to try uploading a bunch of stuff in that time, so if you have a million pending mappings, you might like to just give the PTR a break for a few days. once it has fully caught up, the anonymisation should only be 20-60 seconds of 'busy' a day
- the way the anonymisation works is all serverside services now have a single non-useable 'null account' that will take possession of all content after the delay. the original uploader is lost, and the whole historical record is merged together.
- the privacy help doc has been updated to talk about the new anonymising system. overall, I think the null account pretty much eliminates the speculative account cross-referencing worries we had, and I am happy


Anonymous Board owner 07/07/2021 (Wed) 21:28:13 Id: 76b34f [Preview] No.1101 del
- admin/janny info:
- for admins, all repositories now have an 'anonymisation period' option that you can edit in the service admin menu, defaulting to 90 days (min 1, max 360). you'll also see summary statements in the server logs as updates are anonymised. anonymisation will kick in two minutes after boot, so if you want to change this value immediately on update, get ready and be quick about it!
- for jannies who can see accounts, you will see the null account pop up in reference to older content (moreso in future when I expand janny UI and permissions). it being special is highlighted, and various account modifying UI shows it cannot be edited
- also for jannies/admins, I had to do some wickity woo to get the null account to work without a network update for everyone. if you try to look at the null account on 445 you may get an error. normal users won't run into this, but there's a kind of 'soft' network version update for you today

next week

The null account took most of my time this week, and I now have several extra little jobs to catch up with. So I'll bash away at that small work, and I'd really like to crack the multiple column lists that keep growing in width.


Anonymous 07/12/2021 (Mon) 03:47:44 Id: 181271 [Preview] No.1103 del
>>1092
Thanks for the reply. Great program you've made. It's really revealed that the biggest obstacle to organisation was always me. Even with hydrus I can't be fucked to tag most of my shit.


Anonymous Board owner 07/14/2021 (Wed) 06:41:10 Id: 5c598a [Preview] No.1104 del
>>1103
Yeah, I find it difficult too. The PTR is great, if you can devote the SSD space for it. But you also need to set up workflows and so on just to get through all your inbox. I really haven't figured out the answer there, and for a decade I guess I've slowly seen all my other queues turn infinite too--I add more films every month to my queue than I watch, more vidya than I can play, more music than I can listen to, and more little files through hydrus than I can process.

If everything goes tits up but I end up in a functioning bunker, I'll probably be set for twenty years and finally catch up, ha ha ha.


Release Tomorrow! Anonymous Board owner 07/14/2021 (Wed) 06:41:26 Id: 5c598a [Preview] No.1105 del
I had a great week. I reworked how file status is stored in the database, greatly accelerating many functions all over the program, particularly on very large clients. I also fixed some annoying bugs and added some quality of life.

The release should be as normal tomorrow.


Anonymous 07/15/2021 (Thu) 13:09:06 Id: 7059bb [Preview] No.1106 del
>>1103
the key to tagging is not to tag fully, but work in waves.

mine is
safe / suggestive / explicit - this is a hard gate so if it doesn't have this tag, its not going to the next tag section
real / drawn - another hard gate but its FAR to useful for me to not get this in on the first parse go
funny
favorite
tag further - this implies there is something special there, usually accompanied by favorite, that I need to pay attention to
art
and a few other 'series' gates

from here it gets more specialized. this method lets me tag 1 file every second or so and makes sure all files are 'new' files, so im not wondering about duplicates and shit like that. that said my method relies on me being done with duplicate parsing which is currently at 1.7 million files, so fuck me.

my next method for mass culling rather then tagging is a 10 star system.

with one of the updates I can progressively add ratings with a hotkey, so they start out at 0 star
they move up to 'seen it 0 star' and each time I 'see it' 1 star gets added, ands when files make it to 10 star, they move to archive.

this method was devised due to duplicates takeing FAR to long for me to go through and I wanted to do something else then a/b files. this makes it very easy to go through things quick, and then with collections exclude what I have already seen. granted I have opinions on how that is handled and better ways to potentially do it, but its a good start.


Anonymous 07/18/2021 (Sun) 01:57:50 [Preview] No.1107 del
>>1099
I've been meaning to help out with the PTR. I guess this is a reminder. Is it ok to use tor as a proxy for the ptr? I haven't tried yet but is it blocked or are there any security concerns? I really want to start petitioning these random images that are coming up for some of my tag searches.



Top | Return | Catalog | Post a reply