E

estadiare

autistic coder & right to die activist
Aug 31, 2022
16
I am currently working on an anonymized archive for this site that I can host on tor in case this site ever goes down. As i need a bit of distraction, I've decided to make the search index and the subsequent search engine on my own from scratch :)

i realized that some of my code could also be used to figure out some interesting stats

on this site, there are 900'000 unique words, of which 270'000 are relevant

some examples of the non-relevant words:

- fibroymalgia
- politcial & policitally
- moneypox
- therminologys
- alphafold

even though most recorded words are irrelevant, 95%+ of all total words written on this site are in the relevant category.

most of the "irrelevant" words are typos, scientific terms or terms only used in a certain field, usernames and non-english terms

the most popular word around is "to", with at least one mention in almost any single thread

The longest word ever used is
mjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoieher

as found in this thread:
 
Last edited:
Ginnn

Ginnn

idk
Aug 20, 2022
81
Now "mjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoieher" is my favorite word
 
Angst Filled Fuck Up

Angst Filled Fuck Up

Visionary
Sep 9, 2018
2,125
Now "mjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoiehermjvvianeslsslsfnfnccqpqwodqmdamamcamsfoihfewofhwifgwfhueawprhoierwgievofdnvoieher" is my favorite word
That's a town in Wales, isn't it?
 
C

chronicallybroken

Student
Jul 16, 2022
167
I love this! I find this stuff really interesting. Not the technical side though because I don't understand it :ahhha:
 
locked*n*loaded

locked*n*loaded

Enlightened
Apr 15, 2022
1,995
Why are fibroymalgia and monkeypox irrelevant? If someone suffers from those conditions and it causes them physical pain, and consequentially depression, resulting in them wanting to CTB, how can they be irrelevant? 😕
 
E

estadiare

autistic coder & right to die activist
Aug 31, 2022
16
Why are fibroymalgia and monkeypox irrelevant? If someone suffers from those conditions and it causes them physical pain, and consequentially depression, resulting in them wanting to CTB, how can they be irrelevant? 😕
these are typos, it says moneypox and fibroymalgia
 
locked*n*loaded

locked*n*loaded

Enlightened
Apr 15, 2022
1,995
these are typos, it says moneypox and fibroymalgia
Sorry about that. I guess I'm seeing things that aint there. Don't know how I missed that. Well then, I agree, those words are irrelevant. Carry on.
 
artificial_ineptness

artificial_ineptness

Member
Nov 14, 2021
71
This is pretty cool, I've always spent an inadequate amount of time nerding over data 😅

Do you plan to do more analysis / share some more stats? The first thing that comes to my mind with data like this is to see over- and under- represented words (This Kaggle dataset might be an okay start to be compared with?).

Also, how did you filter out the "irrelevant" words? Maybe you used some dataset of common English words (or maybe just got rid of rare words?)
 
  • Like
Reactions: estadiare
E

estadiare

autistic coder & right to die activist
Aug 31, 2022
16
This is pretty cool, I've always spent an inadequate amount of time nerding over data 😅

Do you plan to do more analysis / share some more stats? The first thing that comes to my mind with data like this is to see over- and under- represented words (This Kaggle dataset might be an okay start to be compared with?).
thank you! Unfortunately i don't plan to do more data analysis for the moment

Also, how did you filter out the "irrelevant" words? Maybe you used some dataset of common English words (or maybe just got rid of rare words?)

for the search engine, I have a list of words and within one word entry there is a list of threads and an associated score. For example, the entry for 'dataset' looks like this (word|thread1:score1,thread2:score2,...):

dataset|11877:4,28438:4,29980:7,30211:2,30739:8,34679:4,43044:0,7608:2,77788:4,82081:29,85404:2,86362:6

so for example in the thread https://sanctioned-suicide.org/threads/n-failure-from-final-exit-question.85404/ the term 'dataset' is only mentioned in one comment so it only has a score of 2

the thread https://sanctioned-suicide.org/threads/status-of-mental-health-2022.82081/ on the other hand has several mentions in the OP post, so it gets a score of 29


my technique for filtering out 'irrelevant' words at the moment is to sum up the scores for a word and if it doesn't reach a certain threshold it is filtered out
 
Last edited: