It’s relatively easy to measure the sheer quantity of users. Or the quantity of time or money they spend on a site. How do you measure user quality?
linked attribution to other sites - genuine engagement that affects the world
Web of trust, with weighted ranking based on people you’re close with. Quality is subjective and depends on the preferences of the person asking. Here is a link that was shared with me by a random anon that I think about a lot: http://comboy.pl/wot.html
Fit a regression model for how historical engagement predicts future engagement. Stack rank on scalar output. Ditto for predicting future sentiment of user post. In both, trying to measure expectation of future “quality” given information set of today.
a relative engagement score which takes into account the engagement scores of the users that engage with your content.
It seems to have become standard practice for platforms to prioritise user engagement metrics like # of users/time spent on the site even when it comes at the expense of the users mental health/general well being Maybe quality is the measure of the presence/absence of these adverse effects relative to engagement
One dimension is the quality of the writing itself. You can assess the vocabulary, the sentiment (too many complaints?) How often they get unfollowed (weighted by quality of the follower perhaps) is an indication of quality. It wouldn't be hard to train a learning model that learns from all these signals.
NPS scores over time. High NPS usually at the start and see how that develops over time. What happens after the honeymoon phase?
We introduce a new vector to measure “thoughtfulness” of content similar to like, retweet, etc. Based on aggregated thoughtfulness over time, we measure a user’s quality. Sounds very crude, but that’s kinda how we do it these days. We assume high likes and high followers equals high quality, but that’s no
Depends on the nature of the product! Ask which aspect of growth is most important: retention, acquisition, or monetization Then determine the actions that best drive retention, acquisition, and monetization
One of the projects we shipped a while ago measured quality in regards to negative engagement (more dwell time meant they weren’t getting the answer/info they needed). Then we tried to surface info as far up funnel as possible (bonus if you could answer it on a Google results page without a needing a page visit)
Potentially, user quality that’s based on number of times a user has engaged and purchased on a site(DAU/MAU) , how many of people they referred to purchase the same products (long term value), and overall influence on the site and/or other social media channels.
in a self-sovereign world users will bring their own algos to process verifiable credentials (signed messages) - quality/trust is in the eye of the behodler
Great question. Up there with how to measure the quality of a user engagement. 1 like = 1 like. It doesn't show the depth of the relationship or the economic value between the two parties involved. Web3 social with embedded financial/social incentives will address this.
Hive.one's algorithmic system for finding reputable accounts within Twitter communities seems like one promising approach https://hive.one/
A repeat distillation process until the app is alcoholic enough
If they login with Ethereum you can peek into their holdings (the flaw here is if they use a throwaway wallet)
I’m bullish on integrating wallets with social activity across the web. If managed right, it can become a de facto Net Promoter Score. It’s a long read, but I break down my take in my personal White Paper https://diamondapp.com/u/BenErsing/blog/an-expanded-future-for-social-tokens-benersing-cc-white-paper-v10
It feels like we’re close to a point where user sentiment (tone of posts or reviews) can be mass analyzed by AI.
Highly contextual for each product. For multiplayer products I found this framework to be useful https://www.carriemelissajones.com/blog/community-commitment-curve?format=amp
This is a key problem for media attribution currently and to evolve legacy monetization models. On chain reputation tied to off chain time/attention/money spent is one way to tackle it.
One way to measure of quality: how much value users can create. Good writing, good code are valuable.
look at how traditional orgs do this, ie Summit requires references so the web of trust replies below hold (to a degree this is what kept Clubhouse early user quality high) so the bigger question is how do you keep user quality high / is eternal september avoidable or a foregone conclusion given enough time
engagement is far too easily gameable, true high value users have illegible qualities that make them high value
Highly context dependent. Quality as defined by what? If engagement, then what kind?
It’s tricky because the answer is different for the different archetypes you need for a successful ecosystem. It’s different for producers/creators vs for users who actively engage with content vs for users who passively consume.
This goes back again to the identity question - like, what do you mean by user? If we were to consider generative nature of identity then identity is irreducible to an identifier. Quality is not static.
retention around the core product mechanics could be one of the ways to measure the quality. (e.g. BeReal: on month 6, do people who stay keep posting what's different enough from Instagram? what % still do?)
So on Blind app, the initial social graph were big tech. Communities often behave like mini societies, they mature over time. One experiment was to open up twitter verification. Same, one-way salted hashed — to see if existing community would give voice to a pseudonymous person with 100k follower on twitter.
big issue with quality atm. solana has millions of wallets, but much more fuzzy on how many are unique users
Wallet activity. A benefit of blockchain is that people can be pressed to put their money where there mouth is. Imagine if Jim Cramers trades were publicly viewable. In VC world imagine NFTs for investments at each stage/ round, anyone can put a logo on their portfolio page but now proof you made the right calls early.
Not sure what instruments of measurement to use but I imagine an approach / method could be derived from the Metaphysics of Quality https://en.m.wikipedia.org/wiki/Zen_and_the_Art_of_Motorcycle_Maintenance
An ugly answer that applies for farcaster: Average lifetime cash value of attached account.